How Amazon, Shopee, and TikTok Shop Recommend Products to a Billion Users

When you visit Amazon's homepage, the site has 200 milliseconds—one-fifth of a second—to pick 20 products from 1.9 billion possibilities, rank them by relevance to you specifically, and display them before you notice a delay. Do this correctly and the user buys something. Do it poorly and they bounce to a competitor.

This is the billion-scale recommendation problem. Three companies—Amazon, Shopee, and TikTok Shop—solved it three completely different ways.

The Core Problem: Too Many Products, Too Little Time

Imagine a physical store with a billion products. A customer walks in and you have one-fifth of a second to pick 20 items they'll like. You can't check everything—there isn't time.

Every major e-commerce platform faces this: - Amazon: 1.9 billion live products (Q4 2025) - Shopee: 500 million products - TikTok Shop: 100 million SKUs

The naive approach—score every product—doesn't work. Even at one microsecond per product, scoring a billion products takes 16 minutes. You need 200 milliseconds.

The solution: break the problem into stages.

Stage 1: Fast Filtering (20–50ms)

Narrow 1 billion products to ~1,000 candidates using cheap approximation.

Stage 2: Careful Ranking (50–100ms)

Apply expensive model to rank the 1,000 candidates using hundreds of signals.

Stage 3: Final Touches (20–50ms)

Apply business rules: diversity, freshness, inventory, margin.

Total: 200 milliseconds.

Amazon: Item Similarity + Semantic Search

Amazon's core idea from 2003: if you liked product A, show you products that other customers bought after viewing A.

How It Works

Precompute Similarity Every night, Amazon compares products based on who bought them together. "People who bought a laptop also bought a mouse, keyboard, laptop bag." When you visit, Amazon looks up similar items for your recent 50 interactions. This takes 5–10ms.

COSMO Semantic Search (2024) Traditional search fails on "shoes for pregnant women." Amazon's COSMO knowledge graph reasons: - "Pregnant women" → "stability needs" - "Stability" → "slip-resistant soles" - Result: show slip-resistant shoes

Zero keyword overlap, but it understands what you need. Adds ~2,000 candidates.

Heavy Ranking Neural network with 1,000+ features scores 27,000 candidates in 50ms on GPUs.

Reranking Final pass adds diversity and business logic.

Total: 115–130ms average, 180–200ms worst case

Why This Works

Amazon optimizes for catalog coverage. With 1.9 billion products, most are "long-tail" with sparse data. COSMO's semantic layer helps rare items get discovered.

Impact: +4–6% clicks on long-tail products (A/B test).

Shopee: Your Current Session Is Everything

Shopee's insight: your current browsing session predicts better than your entire purchase history. Browsing dresses now? You want dress-related items—not the camping gear from six months ago.

How It Works

Build a Session Graph Each product you view is a node, each transition (dress → shoes → bag → dress) is an edge. The graph captures patterns like "users who go dress → shoes often return to dresses."

Graph Neural Network Instead of treating your session as sequence A→B→C, Shopee's graph neural network sees all relationships simultaneously. Captures loops and backtracking.

Dual-Phase Synergistic Attention On product detail pages, two attention branches: - Relevance: Focus on trigger item's category/brand/price - Diversity: Focus on unseen items different from trigger

Multi-Task Ranking Optimizes CTR, conversion rate, and dwell time simultaneously. New users need exploration (diverse categories), loyal users need exploitation (narrow interests).

Total: 63ms average, 95ms worst case

Why This Works

Mobile users browse in short bursts (5–10 minutes, 10–30 views). "What you're doing right now" matters more than old history.

Impact: - Dual attention: +1.8% CTR, +2.1% conversion - Session GNN: +3.2% CTR vs previous baseline - Lifecycle optimization: +2.8% GMV

Cold-Start Advantage

Traditional systems fail on new users (no history) and new products (no purchases). Shopee handles both: - New user: 3 product views are enough to predict the 4th - New product: Use category/brand/visual features

Session-based: 38% recall for new users. Traditional CF: 12% recall.

TikTok Shop: Content and Commerce Are Different

TikTok's radical choice: run two separate algorithms and merge results.

The Key Insight

High engagement ≠ high conversion. The signals differ: - Content algorithm: Watch time, likes, shares—will you keep scrolling? - Commerce algorithm: Purchase rate, sales velocity, return rate—will you buy?

Optimizing one harms the other.

How It Works

Two Parallel Algorithms (30ms each)

Algorithm 1: Content Scoring Transformer over recent 200 videos. Predicts: "Will this user watch and engage?"

Algorithm 2: Commerce Scoring Two-tower model. Predicts: "Will this user purchase?"

Fusion Layer Weighted combination learned per user: - New users: 70% content, 30% commerce - Frequent buyers: 80% commerce, 20% content

Merchant Health Scoring Unlike competitors (filter bad merchants, treat good ones equally), TikTok makes quality a ranking signal: - Fast shipping → boost - Low returns → boost - Poor service → suppress

Aligns incentives: merchants improve to gain visibility.

Niche Consistency Sellers jumping categories get penalized. Focused sellers get +12% reach.

Total: 65–70ms

Why This Works

TikTok is a discovery platform. Users scroll without intent. Dual-stream ensures videos are entertaining AND commercial.

Impact: - Dual-algorithm: +6.2% GMV vs unified model, +8.1% merchant retention - Merchant health: -14% return rate platform-wide - Niche consistency: +12% reach for focused sellers

eBay's Benchmark: 1.7B Items in 25ms

eBay published their retrieval benchmarks:

HNSW Graph Index - Convert each item to 256-number vector - Build graph where similar items connect - Search traverses graph (logarithmic time)

Performance: - 1.7 billion items - <25ms (95th percentile) - 95–98% recall - CPU-only (no GPUs)

Key techniques: Quantization (32-bit → 8-bit) and sharding (100+ machines).

What Drives Business Impact?

Large Lifts (3–6% CTR/GMV)

- Semantic enhancements: COSMO (+4.6% CTR), graph embeddings (+3.2% CTR) - Stratified optimization: TikTok dual-stream (+6.2% GMV), lifecycle modeling (+2.8% GMV)

Small Lifts (1–2%)

- Architectural refinements: attention mechanisms (+1.8% CTR) - Incremental improvements: hyperparameters, more data

Non-CTR Metrics Matter

- TikTok merchant health: small CTR change, -14% returns (huge cost savings) - Shopee lifecycle: +2.8% GMV despite smaller CTR gains

Decision Framework

1. Catalog Size

- <1M: Simple collaborative filtering - 1M–100M: Two-tower + ANN search - 100M–1B+: Hybrid retrieval, multi-stage required

2. Interaction Pattern

- Session-based: Shopee-style graph NN - History-based: Amazon-style CF + semantic - Hybrid: Unified session + profile model

3. Business Objective

- Maximize GMV: Optimize conversion - Maximize engagement: Optimize watch time - Balanced: Dual-stream (TikTok) - Marketplace quality: Merchant health scoring

4. Latency Budget

- <50ms: Single-stage retrieval - <200ms: Two-stage - <500ms: Multi-stage

5. Cold-Start Prevalence

- High: Session-based or content-based - Low: CF with content fallback

Real-World Benchmarks

System	Catalog	Latency (Avg)	Latency (99%)	Stages
Amazon	1.9B	115ms	195ms	5
Shopee	500M	63ms	95ms	3
TikTok	100M	70ms	110ms	2+fusion
eBay	1.7B	15ms	25ms	1

Five Lessons for Leaders

1. Multi-Stage Is Not Optional at Scale

Above 10M items, you must use retrieval → ranking pipeline. Question is how many stages, not whether.

2. Data Beats Models

COSMO (+4.6% CTR) and graph embeddings (+3.2% CTR) beat architectural tweaks. Better data (knowledge graphs, rich features) > better algorithms.

3. Conflicting Goals Need Separation

Clicks vs conversions? Engagement vs retention? Explicit separation (TikTok dual-stream) beats end-to-end training. +6.2% GMV gain.

4. Don't Ignore Non-Click Metrics

CTR is incomplete. TikTok merchant health: minimal CTR impact, -14% returns. Optimize business outcomes, not proxies.

5. Cold-Start Matters

If >20% traffic is new users/products, build cold-start in from the start. Session-based (Shopee) and content-first (TikTok) handle naturally; history-based (Amazon) needs explicit fallback.

What's Next

LLMs Offline, Not Real-Time

GPT-4 too slow (50–200ms) for serving. Use offline: - Generate product attributes - Build knowledge graphs (COSMO pattern) - Create cold-start embeddings

Billion-Parameter Transformers

Recent research: 1B parameters (+2.3% listening time). Challenge: training cost. Solution: train large offline, distill small for serving.

Vector Search for Real-Time

Amazon Valkey: <18ms query-to-recommendation. Enables conversational search ("waterproof boots under $100") without precomputing.

Conclusion

Three different solutions to the same billion-scale problem:

Amazon: Catalog coverage via hybrid retrieval (CF + semantic). COSMO handles zero-overlap queries.

Shopee: Session coherence via graph neural networks. "What you're doing now" > old history. Cold-starts naturally (3 views = enough).

TikTok: Discovery via dual-stream (content + commerce). Solves engagement/conversion conflict that single models can't.

Common pattern: multi-stage pipelines universal at scale. Differences: retrieval strategy and ranking objectives.

Choose based on catalog size, user behavior, business goals, cold-start needs. Invest in data quality first. Model conflicting objectives explicitly. Optimize outcomes, not proxies.

Last updated: July 25, 2026

Sources: Amazon Science (COSMO, Valkey), eBay Tech Blog, Shopee papers (AdKDD 2024), TikTok reports, academic conferences (RecSys, CIKM, WWW, OSDI, NSDI).