AI/ML
A recommender whose most important finding is that recency-weighted popularity wins
A production-style two-stage recommender over 1.3M products, evaluated on a strict global timeline
Sole developer
The problem
Recommenders are easy to over-credit when evaluation leaks future data into training. The goal is a ranked, explained list per user that holds up under time-respecting evaluation, for an Amazon-scale catalogue.
Team: Personal project. Timeline: October 2026.
What changed
- Warm-user HR@20, best source
- 0.024
- Baseline: Two-tower 0.00025, ALS 0.0053
- Recency-weighted popularity on 20,000 warm users, exact full-catalogue ranking, one held-out target each (results/evaluation_electronics_0core.json); every learned model's paired-bootstrap difference against it excludes zero
- Personalisation from fusion
- 6× more distinct top-1 items
- Baseline: 25% lower HR@20
- Shipped fusion of recency popularity with co-occurrence gave 18 distinct top-1 items over 2,000 warm users against 3, at HR@20 0.015 against 0.020 (results/fusion_electronics_0core.json); taken deliberately
- Warm-user latency p95
- 33.0 ms
- Baseline: 150 ms target
- 400 requests at concurrency 8, k=20, one process, 1,334,914 items served (results/load_test_electronics_0core.json); all five latency targets met, 310 requests a second
Architecture
Request flows
- Serve a ranked list: Client → Recommendation API → Retrieval (recency popularity + co-occurrence, fused) → LambdaMART re-ranker and business rules → Template explanations
- Load features: Parquet data and manifests → Embedded online feature store
Decision 1 of 4
How should train and test be split?
- Options
- Leave-one-out per user · Random split · A global timeline on absolute timestamps
- Chose
- One global timeline, with the cut points recorded in every result file and leakage tested automatically
- Trade-off
- Numbers are lower and harder to compare with papers that use leave-one-out.
- Evidence
- 57% of test users have no training history and 41% target an item unseen at training time, which a per-user split would hide.
Decision 2 of 4
What happens when the learned models lose?
- Options
- Drop them quietly · Tune until they win · Report the loss and ship what won
- Chose
- Report the loss and ship what won
- Trade-off
- The headline result is a negative one.
- Evidence
- Two-tower retrieval scored HR@20 0.00025 and the LambdaMART re-ranker scored −0.0173 against its own stage 1; the hand-written rule layer changed no hit rate at all.
Decision 3 of 4
Accuracy or personalisation?
- Options
- Serve recency-weighted popularity alone · Fuse it with item-to-item co-occurrence
- Chose
- Reciprocal-rank fusion of the two
- Trade-off
- HR@20 falls by 25% (paired difference −0.005, 95% CI −0.0095 to −0.0010).
- Evidence
- Fusion gives 18 distinct top-1 items across 2,000 warm users instead of 3, so users stop seeing the same few products.
Decision 4 of 4
Should a language model write the explanations?
- Options
- Generate explanations with a language model · Deterministic templates from the evidence the model used
- Chose
- Deterministic templates
- Trade-off
- Explanations are less varied.
- Evidence
- Every explanation is filled from evidence the ranking actually used, so it can't describe a reason that wasn't there.