The X algorithm, reverse-engineered in plain sight.
A visual, interactive walkthrough of the open-sourced xai-org/x-algorithm stack — Home Mixer, Thunder, Phoenix and the candidate pipeline. Read the code, tune the weights, watch the feed re-rank in real time.
Nine stages from post to feed.
The Home Mixer assembles your For You feed through a strictly ordered sequence. Sources run in parallel, hydrators enrich, filters strip, scorers rank, and a blender finally weaves ads and modules into the organic stream.
Query hydration
21 parallel hydrators load the viewer's followed accounts, mutual-follow MinHash, impression bloom filters, served history, blocked users, muted keywords, demographic context and the user's recent engagement sequence.
- FollowedUserIdsHydrator
- ImpressionBloomFilterHydratorfast 'seen?' lookups
- UserActionSeqHydratorengagement history → Phoenix
- ServedHistoryHydrator
- BlockedUserIdsHydrator
- MutualFollowQueryHydrator
The full journey of a single post.
Every post you read in the For You feed travelled through this. From a Kafka message at the moment of creation to a ranked, blended item in someone's timeline — usually in under a few hundred milliseconds.
Author posts. Thunder eats the Kafka event and stores it for sub-ms lookups.
Realtime in-memory store. Original posts, replies, video posts kept per-user.
Candidate tower embedding stored for ANN retrieval over the global corpus.
Home Mixer hydrates the viewer's context: follows, mutes, MinHash, history.
Thunder returns in-network candidates; Phoenix returns out-of-network candidates.
Add author info & engagement counts. Strip blocked, muted, stale, duplicates.
Transformer predicts P(action) for every candidate × 19 actions.
Σ wᵢ·P(actionᵢ), attenuate repeated authors, slot ads, return top 250.
In-network and out-of-network, in parallel.
Two completely separate retrieval systems run side by side and feed their results into the same ranker. One is a realtime in-memory store. The other is a transformer-based nearest-neighbour search over the global post corpus.
Realtime in-memory store of posts from accounts you follow.
Thunder consumes post create/delete events from Kafka and keeps per-user collections for original posts, replies, reposts and video posts. Old posts are trimmed automatically. Lookups for in-network content happen in sub-millisecond time without ever hitting a DB.
- SourceKafka topic of post events
- StoragePer-user buckets, trimmed by age
- Latency<1 ms median lookup
- Recall100% of follows · recent posts only
1let request = GetInNetworkPostsRequest {2 user_id: query.user_id,3 following_user_ids: following_list,4 max_results: ThunderMaxResults,5 exclude_tweet_ids: query.seen_ids,6 algorithm: ThunderAlgorithm,7};8// returns recent posts from people you follow9let response = client.get_in_network_posts(request).await?;Two-tower transformer that finds posts you haven't seen yet.
A user tower encodes your engagement history into one embedding; a candidate tower has already encoded every post in the global corpus. Approximate nearest-neighbour search by dot product returns the top hundreds of candidates outside your follow graph.
- ArchitectureTwo-tower, Grok-based transformer
- Embedding128-dim · multi-hash lookups
- Index~537K-item demo corpus · prod ≫
- OutputTop-K candidates by cosine
1def retrieve(user_features, history, corpus_embeddings, k=200):2 # User tower encodes the viewer + their recent actions3 user_emb = user_tower(user_features, history) # [B, D]4 # Candidate tower has pre-encoded the corpus5 scores = jnp.matmul(user_emb, corpus_embeddings.T) # [B, N]6 top_k = jnp.argsort(-scores, axis=-1)[:, :k] # [B, k]7 return top_kIn-network sourcing guarantees coverage of people you've chosen to follow. Out-of-network sourcing prevents the feed from collapsing into a small clique — and it's the only way new creators ever reach you. Production weights the two with the OonWeightFactor multiplier (educational default 0.75).
Every engagement has a price. Some are wildly more expensive than others.
Phoenix predicts the probability of each action and a weighted sum becomes the score. A follow is worth roughly 100× a passive dwell, and a single report drops your score by hundreds of times more than a like adds.
- Report-369
- Block author-120
- Mute author-100
- Not interested-74
- Follow author+50
- Reply+27
- Profile click+12
- Share → DM+2.5
- Not-dwelled-2
- Share+1.5
- Quote+1.2
- Share → Copy link+1.2
- Repost+1
- Favorite+0.5
- Link click+0.12
- Quoted click+0.08
- Photo expand+0.03
- Dwell time+0.01
- Video quality view+0.005
- Dwell+0.001
Each row above represents one of the action probabilities Phoenix predicts. The bar shows the weight applied to that probability before summing into the final score.
final_score = Σᵢ wᵢ · P(actionᵢ)
↑ ↑
weight probability
the Phoenix
transformer
predictsDistribution unlocks in stages.
Virality on X isn't a single event. It is a chain of unlocks where engagement at one tier qualifies the post for the next, larger pool. Each tier amplifies the reach by orders of magnitude.
Each tier doesn't add reach — it multiplies the pool Phoenix considers for retrieval. Profile clicks and follows are the cleanest viral catalysts because the model treats them as evidence the author is worth long-term re-exposure.
A 2-second dwell on a long video weighs more than ten silent likes. The cont_dwell_time signal is one of the highest-impact passive metrics.
A reply costs effort. Phoenix correctly treats it as harder evidence of interest. Replies also keep your post pinned to the conversation tree, which DedupConversation rewards.
A follow during a single session is the rarest, strongest positive. It signals 'I want this author in my feed permanently.' Subsequent posts get boosted on retrieval AND scoring.
Posts get cheaper as they age.
Three mechanisms quietly suppress reach: the hard 36-hour age filter kills old posts entirely; freshness reduces effective score; and author-diversity attenuation drops your second, third and fourth post per feed by a geometric ratio.
Score halves every ~90 minutes.
The For You feed strongly prefers fresh content. Past 24h, scores have already dropped to a fraction of their peak. Past 36h, the AgeFilter kills the candidate entirely before scoring even runs.
Your second post per feed already pays a tax.
When the same author has multiple candidates in a feed, the second is multiplied by 0.7, the third by 0.525, then 0.46, then 0.43… asymptoting to the 0.4 floor. Posting back-to-back hurts your individual scores.
AgeFilter drops anything older.
Multiplier = (1 − 0.4) · 0.5^pos + 0.4
Shorter videos lose the VQV bonus entirely.
Every post starts on a small bench.
The ranker tests new posts against a tiny audience first. Pass that test, and Phoenix expands the audience — fail it, and the post gets quietly retired regardless of who the author is.
First exposure to an in-network slice. The Phoenix ranker decides if the post even deserves to live past this tier based on the early dwell, replies and like-rate.
Thunder pushes the post into more in-network feeds. If reply rate stays above the author's baseline, it qualifies for OON Phoenix retrieval.
Phoenix retrieval starts including the post for users whose engagement embedding is close to the post's. Profile clicks at this tier matter more than reposts.
Network effects compound. Each tier-2 follow re-ranks all of that user's future feeds toward the author. Reach now grows non-linearly.
The fastest way to disappear.
Suppression on X is not a single shadowban switch. It is many quiet, cumulative penalties — some scored, some filtered, some applied long before your post ever reaches the ranker.
Strongest single negative on the author. Subsequent posts from them get a discounted score in your feed and your follow-graph features quietly de-weight them globally.
Triggers visibility filtering at the VFFilter stage. If enough reports cluster, the post can hit Action::Drop and never appear, regardless of score.
Soft block. AuthorSocialgraphFilter strips their candidates entirely for you, and your action sequence carries the negative for future authors of similar style.
User-facing 'show me less' tap. Phoenix learns this as a labelled negative and downstream similar posts get scored lower.
Implicit negative for posts the user scrolls past without stopping. Individually small, but accumulates fast.
These don't lower your score — they delete the candidate.
Visibility filters and policy filters run as kill-switches, not re-rankers. If your post matches one, no amount of engagement can push it back into anyone's feed.
Removes posts the visibility-filtering system has marked spam, violence, gore or deleted. Anything with Action::Drop on its safety result is gone before the user ever sees it.
Tokenizes both your muted keywords and the post text via TweetTokenizer. Any match removes the candidate. Acts pre-score so muted topics never enter ranking at all.
Drops posts whose author you've blocked or muted — and whose author has blocked you. Also drops quoted/retweeted users that you've blocked.
Paywalled / subscription-locked content that the viewer can't access. Removed pre-score so the model never wastes capacity ranking it.
Tiny edits, huge score deltas.
Three pairs of nearly-identical posts. The Phoenix predictions shown below are the same model output across both variants for the same demo viewer — only the post changed.
I've been thinking a lot lately about how the way we measure engineering productivity is completely broken in most companies — most teams optimize for output, not impact, and you end up with people shipping a lot of low-quality features instead of doing the deep, slow work that actually moves a product…
Most engineering productivity metrics measure activity, not impact. Here's a better one. ↓
Hookless openings never trigger dwell. A single tight sentence pulls dwell_time up 4× and drags the reply rate with it.
Glossary — words you'll keep meeting.
The Rust orchestration service that runs the candidate pipeline and returns a ranked feed via gRPC. Owns the For You endpoint.
In-memory realtime store of recent posts from accounts you follow. Serves in-network candidates in sub-ms via the InNetworkPostsService.
Grok-based transformer system that handles two things: retrieval (two-tower model) and ranking (transformer with candidate isolation attention).
A reusable Rust framework defining traits — Source, Hydrator, Filter, Scorer, Selector, SideEffect — that compose into recommendation pipelines.
Video Quality View — a binary 'meaningful watch' label produced only for videos longer than MIN_VIDEO_DURATION_MS (10 seconds).
Out-of-network multiplier applied after scoring. Defaults to 0.75 to slightly penalise OON posts; bumped to 1.0 for new users with few follows.
Geometric decay applied to repeated authors in a feed. Multiplier = (1−floor) · decay^pos + floor. Production defaults: decay=0.5, floor=0.4.
A probabilistic 'has the user seen this?' set delivered with every request. Pre-score filter strips anything that may have already been served.
256-bit signature stored per user. The MutualFollowJaccardHydrator computes overlap with the candidate author's signature — a fast 'shared social graph' score.
Visibility filter — a policy stage that drops candidates whose safety result is Action::Drop (spam, gore, violence, deletes).
Content-understanding service for spam detection, post-category classification and PTOS policy enforcement. Feeds VF and brand-safety stages.
Determinantal Point Process — diversification used optionally inside the VMRanker with theta and max_selected_rank parameters.
Turn the dials. Watch the feed rearrange.
Open the simulator to see how a real ranking pipeline reorders an example feed when you change Phoenix weights, toggle author diversity, or shift the OON multiplier.