Recommender Evaluation

This skill defines how the Vinyl Record Store recommender is measured. It exists because recommender quality is judged on ranking and catalog health, not MSE/RMSE — applying regression metrics to a top-k recommender is a classic, grade-costing mistake.

When to use

• You are about to compute or report a number about recommendation quality.
• You are designing the train/test split or deciding what counts as "relevant."
• You are comparing two algorithms and need a fair, side-by-side table.
• You are writing the Evaluation section of a CSX4207 report or slide.

Step 0 — Define "relevant" before touching metrics

Pin this down explicitly and write it in the report:

• Explicit ratings: "relevant" usually = rating ≥ threshold (e.g., ≥ 4 of 5).
• Implicit feedback: "relevant" = user interacted (play/purchase) in the held-out period; for ranking metrics, consider only items the user hasn't already consumed from training.

Ambiguity here invalidates every downstream number.