session-feedback-analyzer
Session Feedback Analyzer
Mines Claude Code session JSONL for implicit user feedback. When a user corrects, redoes, reverts, or partially accepts AI output after a skill invocation, that signals a skill gap. Outputs structured feedback.jsonl with per-event dimension attribution for the improvement pipeline.
When to Use
- Compute per-skill correction rates to find which skills users correct most often.
- Generate
feedback.jsonlas input for improvement-generator's candidate prioritization. - Track correction trends over time (30-day rolling windows) to detect skill quality regression.
- Identify hotspot dimensions (accuracy vs coverage vs trigger_quality vs efficiency) per skill.
- Compare correction_rate before and after an improvement to validate whether a change actually helped.
- Audit a single skill's feedback history with
--skill-filterto understand why users reject its output. - Feed dimension hotspots into improvement-generator so candidates target the dimensions users care about.
- Bootstrap the auto-improvement loop: analyzer output is the starting signal that tells the pipeline which skills need work.
- Investigate spikes in correction_rate after a skill update to decide whether to rollback.
When NOT to Use
- Synthetic task evaluation against a predefined task suite -- use improvement-evaluator instead.
More from lanyasheng/auto-improvement-orchestrator-skill
skill-distill
|
1improvement-gate
当执行完变更需要验证是否应保留、候选被标记 pending 需要人工审批、或想查看待审队列时使用。6 层机械门禁: Schema→Compile→Lint→Regression→Review→HumanReview，其中 Schema/Compile/Regression/Review 为阻塞层（失败即拒绝），Lint 和 HumanReview 为建议层（失败不阻塞但记录警告）。不用于打分（用 improvement-discriminator）或执行变更（用 improvement-executor）。
1prompt-hardening
硬化 agent prompt、system prompt、SOUL.md、AGENTS.md、cron prompt 使 LLM 可靠遵循指令。触发词：agent 不听话、忽略规则、绕过约束、prompt 优化、指令合规、规则强化、prompt 硬化、LLM 不遵守、模型违规、creative circumvention。Use when agent ignores rules, disobeys instructions, bypasses tool constraints, needs prompt optimization, instruction compliance improvement, or rule hardening. 不适用于代码生成、代码审查、测试编写等执行型任务。参见 improvement-orchestrator (用于 skill 质量改进)、code-review-enhanced (用于代码审查)。
1benchmark-store
当需要初始化基准数据库、对比 skill 评分与历史基线、查看 Pareto front 是否有维度回退、或查阅质量分级标准时使用。不用于给候选打分（用 improvement-discriminator）或自动改进（用 improvement-learner）。
1skill-forge
>
1improvement-evaluator
当需要验证 Skill 改进是否真正提升了 AI 执行效果时使用。通过预定义任务集（YAML）运行 AI 任务，判定 pass/fail，输出 execution_pass_rate。不用于文档结构评分（用 improvement-learner）或候选打分（用 improvement-discriminator）。
1