Build an AI Content Sorter

Build an AI sorter with DSPy: define categories, load data, evaluate, optimize, and deploy.

Step 1: Define the sorting task

Ask the user:

1. What are you sorting? (tickets, emails, reviews, messages, comments, etc.)
2. What are the categories? (list all labels/buckets)
3. One category per item, or multiple? (e.g., "priority" vs "all applicable tags")
4. Do you have labeled examples already? (a CSV, database, spreadsheet with items + their correct category)

The answers determine which pattern to use below.

When NOT to use AI sorting

• Categories are deterministic — if you can write regex or keyword rules that cover 95%+ of cases, skip the LM. A message.contains("invoice") rule is faster, cheaper, and more predictable than an LM call.
• You need exact reproducibility — LM outputs can vary between runs. If identical inputs must always produce identical outputs (e.g., for compliance), use rule-based logic or pin temperature=0 and accept minor model-version drift.
• Binary filtering with clear signal — spam filters where a blocklist or Bayesian filter already works well do not need an LM.