CLIP Fine-tuning Recipes Skill

Skill by ara.so — Devtools Skills collection.

Overview

clip-finetune-recipes is a production-ready toolkit for fine-tuning CLIP-style dual encoders on custom image-text data. It emphasizes practical hygiene: data leakage prevention, proper contrastive batch construction for multi-GPU training, and stable learning rate schedules that preserve temperature parameters.

Key features:

• Streaming data pipeline with webdataset shards and automatic deduplication
• DDP training with proper local-loss aggregation across GPUs
• LoRA, full fine-tuning, and linear probe training modes
• Hard-negative mining for both text and image sides
• Evaluation hooks for zero-shot classification and retrieval tasks
• Sanity checks for temperature drift, gradient norms, and embedding collapse

Installation