abtesting-design

Purpose

This skill enables precise A/B test design for OpenClaw, covering hypothesis formulation (e.g., null and alternative), setup of control/variant groups, hash-based randomization, and stratification to ensure balanced experiments.

When to Use

Use this skill when designing experiments for feature comparisons, such as testing website layouts, app features, or marketing campaigns, to validate hypotheses with statistical rigor and minimize bias.

Key Capabilities

• Formulate hypotheses: Define null (e.g., "No difference in click rates") and alternative (e.g., "Variant increases clicks by 10%").
• Set up groups: Configure control and variant setups with parameters like sample sizes and metrics.
• Implement randomization: Use hash-based methods (e.g., SHA-256 on user IDs) for assignment to reduce selection bias.
• Apply stratification: Divide users into strata (e.g., by demographics) to balance groups, using algorithms like stratified sampling.

Usage Patterns

Always start by defining your hypothesis and groups. Use CLI for quick designs or API for programmatic integration. Provide all required inputs (e.g., hypothesis strings, variant names) in a single command or request. For repeated use, store configurations in JSON files and reference them via flags. Validate inputs before execution to avoid runtime errors.