benchmark-framework
Cognitive Skills Benchmarking Framework
Overview
A rigorous framework for A/B/C testing reasoning patterns to empirically determine which cognitive methodologies perform best across problem categories. This framework enables data-driven pattern selection rather than heuristic-based choices.
Why Benchmark?
Different reasoning patterns (ToT, BoT, SRC, HE, AR, DR, AT, RTR, NDF) claim different strengths, but without empirical measurement:
- We cannot validate these claims
- We cannot quantify trade-offs (quality vs. cost vs. time)
- We cannot track improvement over time
- Pattern selection remains subjective
This framework provides scientific rigor to cognitive skill evaluation.
Benchmark Structure
More from kimasplund/claude_cognitive_reasoning
self-reflecting-chain
Sequential reasoning with deep self-reflection and backtracking. Use when problems have step-by-step dependencies, need careful logical reasoning, or require error correction. Each step includes self-reflection, and incorrect steps trigger backtracking. Ideal for debugging, mathematical proofs, sequential planning, or causal analysis where order matters.
96agent-memory-skills
Self-improving agent architecture using ChromaDB for continuous learning, self-evaluation, and improvement storage. Agents maintain separate memory collections for learned patterns, performance metrics, and self-assessments without modifying their static .md configuration.
44chromadb-integration-skills
Universal ChromaDB integration patterns for semantic search, persistent storage, and pattern matching across all agent types. Use when agents need to store/search large datasets, build knowledge bases, perform semantic analysis, or maintain persistent memory across sessions.
42integrated-reasoning
Meta-orchestration guide for choosing optimal reasoning patterns. Analyzes problem characteristics and recommends which cognitive methodology to use - tree-of-thoughts (find best), breadth-of-thought (explore all), self-reflecting-chain (sequential logic), or direct analysis. Use when facing complex problems and unsure which reasoning approach fits best.
30document-writing-skills
Teaches document writing patterns and templates that agents apply when generating documentation, reports, contracts, guides, and technical writing. Use when creating API docs, user guides, reports, changelogs, ADRs, or technical documentation.
23error-handling-skills
Universal error handling, exception management, and logging best practices for all development agents across JavaScript/TypeScript, Python, Rust, Go, and Java. Use when implementing error handling, exception management, logging, error recovery, or debugging production issues.
20