long-horizon-holdout
The Long-Horizon Holdout Protocol
Overview
A measurement framework that challenges short-term growth wins by maintaining long-term control groups (holdouts) to verify if immediate lifts translate to sustainable value over 1-3 years.
Core principle: 30-40% of short-term "wins" show neutral or zero impact long-term.
The Process
┌─────────────────────────────────────────────────────────────────┐
│ T-0 (Launch) │
│ Split Traffic: 90% Treatment / 10% Long-term Holdout │
├─────────────────────────────────────────────────────────────────┤
│ T+3 Weeks │
│ Initial Decision: Ship if short-term positive/neutral │
│ Keep holdout running │
├─────────────────────────────────────────────────────────────────┤
More from coowoolf/insighthunt-skills
gamification-triad
Use when designing retention mechanisms, habit loops, or auditing why users drop off despite engaging with core features, to structure gamification beyond superficial badges
46gardening-mindset
Use when dealing with ecosystems, network effects, or high-uncertainty environments where the right answer cannot be known in advance, when rigid planning consumes more value than it creates
45minimum lovable product (mlp)
In an era where AI lowers the cost of building software, viability is obsolete. The differentiator is joy and emotional connection. Prioritize "Wow" over "Aha"—brand is product interaction.
16three-layer-agent-stack
Use when building AI-powered products or agents, when raw model intelligence isn't enough to solve user problems, or when designing the architecture for agentic workflows
15pre-mortem-kill-criteria
Use before launching products or signing contracts, when needing to combat sunk cost fallacy, or when standard pre-mortems fail to change behavior
14curiosity-loops
Use when facing a significant decision (career pivot, product direction, technical choice) and feeling stuck or indecisive, when seeking contextual advice rather than generic recommendations
13