Preference Optimization

This skill assumes finetuning-method-selection already routed here because the data shape is preference pairs or unpaired thumbs-up/down feedback, not demonstrations (that's lora-qlora-recipes) or a verifiable reward signal (that's grpo-rlvr-training). What follows is method selection among the DPO family, the evidence for how much that selection actually matters, the production training pattern, and how to build the pairs in the first place.