MarkovScale: Optimal Sequential Scaling at Inference Time

This skill enables Claude to design and implement principled sequential scaling systems for LLM inference pipelines. Instead of using ad-hoc retry counts or heuristic refinement loops, it applies MarkovScale's framework: model the retry/refinement process as a two-state Markov chain (Correct vs. Wrong), estimate transition probabilities from a small calibration set, then compute closed-form accuracy bounds and optimal stopping criteria. This replaces guesswork with mathematically grounded decisions about when additional LLM calls help, when they're neutral, and when they actively degrade quality.

When to Use

• When the user is building an LLM pipeline with retry or self-refinement loops and asks how many iterations to run
• When implementing Best-of-N sampling, majority voting, or sequential refinement and wanting to optimize the compute budget
• When the user asks to reduce inference costs on a pipeline that makes repeated LLM calls for the same query
• When designing an evaluation harness that needs to determine if sequential attempts are actually improving accuracy
• When building an agentic system where a model retries failed tool calls or code generation and the user wants principled stopping criteria
• When comparing parallel scaling (independent samples) vs. sequential scaling (iterative refinement) strategies

Key Technique

The Two-State Markov Model. MarkovScale treats each round of sequential LLM inference as a transition in a two-state Markov chain. The states are Correct (C) and Wrong (W). Four transition probabilities govern the system: p_cc (correct stays correct), p_cw (correct flips to wrong), p_wc (wrong flips to correct), and p_ww (wrong stays wrong). Because each row sums to 1, only two free parameters matter: p_cw (the "corruption rate") and p_wc (the "repair rate"). This compact model captures the essential dynamics of sequential refinement.

Closed-Form Bounds. Given an initial accuracy a_0 and the transition probabilities, accuracy after n rounds of sequential scaling can be expressed in closed form using the Markov chain's stationary distribution and convergence rate. The critical insight is the relationship between p_wc and p_cw: when p_wc > p_cw (the model is more likely to fix errors than introduce them), sequential scaling provably improves accuracy and converges to an upper bound of p_wc / (p_wc + p_cw). When p_wc = p_cw, additional rounds have zero net effect. When p_wc < p_cw, further iterations degrade accuracy. This gives an analytical stopping criterion: continue only while the expected marginal accuracy gain exceeds a cost threshold.