skill-creator
Skill Creator
A skill for creating new skills and iteratively improving them.
At a high level, the process of creating a skill goes like this:
- Decide what you want the skill to do and roughly how it should do it
- Write a draft of the skill
- Create a few test prompts and run claude-with-access-to-the-skill on them
- Help the user evaluate the results both qualitatively and quantitatively
- While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify if you feel something needs to change about them). Then explain them to the user (or if they already existed, explain the ones that already exist)
- Use the
eval-viewer/generate_review.pyscript to show the user the results for them to look at, and also let them look at the quantitative metrics
- Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks)
- Repeat until you're satisfied
- Expand the test set and try again at larger scale
Your job when using this skill is to figure out where the user is in this process and then jump in and help them progress through these stages. So for instance, maybe they're like "I want to make a skill for X". You can help narrow down what they mean, write a draft, write the test cases, figure out how they want to evaluate, run all the prompts, and repeat.
On the other hand, maybe they already have a draft of the skill. In this case you can go straight to the eval/iterate part of the loop.
More from s-hiraoku/vscode-sidebar-terminal
vscode-extension-expert
This skill provides expert-level guidance for VS Code extension development. Use when implementing new extension features, debugging extension code, designing WebView UIs, implementing Language Server Protocol features, or optimizing extension performance. Covers activation events, contribution points, VS Code API patterns, security best practices, testing strategies, and publishing workflows.
73vscode-webview-expert
This skill provides expert-level guidance for implementing VS Code WebView features. Use when creating WebView panels, implementing secure CSP policies, handling Extension-WebView communication, managing WebView state persistence, optimizing WebView performance, or debugging WebView rendering issues. Covers security best practices, message protocols, and VS Code-specific WebView patterns.
30mcp-deepwiki
This skill provides guidance for using DeepWiki MCP to explore GitHub repository documentation. Use when researching repository structure, understanding library APIs, asking questions about open-source projects, or exploring documentation for any GitHub repository.
25vscode-extension-refactorer
This skill provides expert-level guidance for refactoring VS Code extension code. Use when extracting classes or functions, reducing code duplication, improving type safety, reorganizing module structure, applying design patterns, or optimizing performance. Covers systematic refactoring workflows, code smell detection, safe transformation techniques, and VS Code-specific patterns.
23vscode-tdd-expert
This skill provides expert-level guidance for Test-Driven Development (TDD) in VS Code extension development following t-wada methodology. Use when writing tests before implementation, creating comprehensive test suites, implementing Red-Green-Refactor cycles, or improving test coverage for extension components like WebViews, terminal managers, and activation logic.
22vscode-extension-debugger
This skill provides expert-level guidance for debugging and fixing bugs in VS Code extensions. Use when investigating runtime errors, fixing memory leaks, resolving WebView issues, debugging activation problems, fixing TypeScript type errors, or troubleshooting extension communication failures. Covers systematic debugging workflows, common bug patterns, root cause analysis, and prevention strategies.
22