Duolingo LLM Feature Engineering

Getting a model to do something once is a demo. Getting it to do that thing on the ten-thousandth call, for a user you have never met, without anyone noticing that quality slid — that is the work this skill covers.

Distilled from Duolingo's public engineering and product writing across test-item generation, audio content at scale, conversational practice and large-scale code migration. The reader is assumed to be building something else entirely; every node ends in questions about your product.

Scope note. Whether to build the AI feature at all, where in the business to point AI, what it does to unit economics, and how to divide labour between your expert org and your models across a whole pipeline belong to the duo-ai-product-strategy skill. This one starts after that decision and is about making the thing not break.

Prompt structure — the artifact, not the message

• [[references/prompt-as-template-not-freeform]] — fixed constraints in one zone, per-instance slots in the other, so operators supply judgement and never format.
• [[references/examples-from-your-corpus-beat-more-instructions]] — when output is poor, add curated in-domain exemplars and delete rules; instructions describe the target, examples are it.
• [[references/find-the-context-optimum]] — prompt size has a peak, not a budget; the worked example earns its place by frequency.

Decomposition — one job per call

• [[references/one-narrow-prompt-per-move]] — build a conversational feature as a small state machine with a mid-flight override and a programmed ending, not one long free-running prompt.
• [[references/isolate-the-high-stakes-generation]] — split the critical artifact into its own call, because constraint competition sacrifices the rule the feature exists for.
• [[references/memory-as-extracted-facts-not-transcripts]] — extract a bounded, typed fact list at the session boundary instead of replaying history forever.