Skill Creator

A skill for creating new skills and iteratively improving them.

At a high level, the process of creating a skill goes like this:

• Decide what you want the skill to do and roughly how it should do it
• Write a draft of the skill
• Create a few test prompts and run claude-with-access-to-the-skill on them
• Help the user evaluate the results both qualitatively and quantitatively
  • While the runs happen in the background, draft some quantitative evals if there aren't any. Then explain them to the user
  • Use the eval-viewer/generate_review.py script to show the user the results, and also let them look at the quantitative metrics
• Rewrite the skill based on feedback from the user's evaluation
• Repeat until satisfied
• Expand the test set and try again at larger scale

Your job is to figure out where the user is in this process and help them progress. Maybe they want to make a skill from scratch, or maybe they already have a draft and want to iterate.

Be flexible -- if the user says "I don't need to run a bunch of evaluations, just vibe with me", do that instead.