magic-linguistic-discourse
When to Use
- Long-context LLM eval where coherence matters (summarization, multi-paragraph QA, RAG).
- Coreference annotation or eval, especially for pro-drop languages.
- Choosing between discourse-annotation frameworks (RST vs PDTB vs SDRT vs GUM).
- Diagnosing model failures: hallucinated references, dangling pronouns, topic drift, broken citation.
When NOT to use: purely sentence-level eval → magic-linguistic-eval. Syntactic structure → magic-linguistic-syntax. Sense-level meaning → magic-linguistic-semantics.
Stance
Discourse is the layer most LLM evals don't touch — and where modern LLMs most often quietly fail. A model can fluently produce a 2,000-word answer with a hallucinated citation, an unreachable referent, or a topic that drifts from question to claim to anecdote without anyone noticing. Discourse-aware analysis catches this.
This is a framework-application skill, not a procedural recipe. Different frameworks model different aspects of discourse coherence. Pick the one that matches your question. Apply it as a lens, not as a script. (Stede's textbook captures this stance — it surveys frameworks rather than prescribing one.)