MDL: Unified Multi-Distribution Learner for Recommendation Systems

This skill enables Claude to design, implement, and advise on Multi-Distribution Learning (MDL) architectures for large-scale recommendation systems. MDL treats scenario information (e.g., different product surfaces) and task information (e.g., click, like, share prediction) as learnable tokens that "prompt" a shared deep network -- analogous to how prompt tokens steer LLMs. This replaces traditional gating or tower-splitting approaches (MMoE, PLE, STAR) with a unified tokenize-and-interact framework that achieves stronger feature utilization and better joint modeling of scenarios and tasks.

When to Use

• When the user needs to build a ranking/recommendation model that serves multiple scenarios (e.g., homepage feed, search results, detail page) from a single architecture
• When implementing multi-task learning for recommendation (predicting click, conversion, like, share, etc. simultaneously) and existing shared-bottom or mixture-of-experts approaches plateau
• When the user wants to replace scenario-specific towers or gating networks with a more parameter-efficient attention-based design
• When scaling a recommendation model to billions of parameters and needing scenario/task signals to interact deeply with features rather than only at the output layer
• When designing a feature interaction module that goes beyond DCN or DeepFM by treating grouped features as tokens with self-attention
• When migrating from separate per-scenario models to a unified model and needing a principled architecture

Key Technique

The core insight is that scenario IDs and task IDs should not be shallow signals (gate inputs, tower selectors, or bias terms). Instead, MDL converts them into dense token representations that participate in the same attention-based interaction pipeline as feature tokens. This "Tokenize-and-Interact" philosophy is borrowed from how LLM prompts steer model behavior: scenario and task tokens act as persistent prompts that activate relevant subspaces of the model's parameters at every layer.

Three stacked mechanisms form each MDL block: (1) Feature Token Self-Attention -- grouped features are projected into fixed-dimension tokens and interact via self-attention (with TokenMixing + per-token FFN), capturing high-order feature crosses without explicit cross-network design. (2) Domain-Feature Cross-Attention -- task tokens query against feature token keys/values, allowing each task to selectively attend to the features most relevant to its prediction objective. Scenario tokens undergo an analogous process. (3) Domain-Fused Aggregation -- for each instance, the active scenario tokens (instance-specific + global) are mean-pooled and added into task token representations, fusing scenario context into task predictions without separate scenario-specific heads.