generate-judge-problems

Installation
SKILL.md

Generate judge v2 problems

Read these references before authoring any file:

Generate v2 problems only: the statement file is exactly problem.md inside the problem directory. Never generate the legacy <id>.problem.md naming.

For an agent challenge (a problem the learner solves by having an AI agent call an Exercode playground API; see agentChallenge in references/problem-format.md), skip steps 2–9 and the stdin/stdout, test-case, ## 制約, template and code-rule rules of references/problem-authoring.md, which assume a submitted program; its difficulty levels and its other statement rules (only concepts the learner has studied, concise unambiguous Japanese) still apply. Write problem.md with the agentChallenge frontmatter and a Japanese statement without ## 制約 / ## 入力 / ## 出力 sections: name the technique to practise and the playground API to use, and end with a ## 正解条件 section. Write the explanation (the solution, an example prompt, and common mistakes) in model_answers/explanation/explanation.md, then validate with step 10 and report with step 11.

  1. Confirm the inputs: learning objectives or the source material, target course, target programming language(s), number of problems, and difficulty range. Place each problem under <courseId>/problems/<problemId>/; never create a repository-global problem or reference another course's problem. Derive problem drafts (id, name, overview, objective, difficulty) from the inputs; when generating multiple problems, order them from easy to hard per the difficulty levels in references/problem-authoring.md.
  2. Make the problem files reproducible from the repository. Preserve existing mise configuration and compatible pinned versions. If none of mise.toml, .mise.toml, or .tool-versions exists, create .tool-versions; otherwise keep the existing format. Add exact versions of every runtime, package manager, and command-line tool required to execute the tracked model answers, templates, judges, and debuggers. Declare their imported third-party libraries in each language's standard project manifest with exact direct versions when supported, and update its standard lockfile. Run mise trust --yes for a mise TOML config, run mise install, and then use the installed commands normally. Do not rely on global installations, ad hoc downloads, or temporary dependency environments, and do not add tools used only to author or validate the problem.
  3. For each problem, write the primary model answer FIRST under model_answers/<languageId>/ (e.g. python/main.py). Solving the problem yourself before writing the statement exposes ambiguities early. Use only concepts the learner has already studied; when no studied-concept list is given, derive reasonable assumptions from the learning objectives and state them in the final report.
  4. Write problem.md (frontmatter + Japanese statement) per references/problem-format.md, following the statement rules in references/problem-authoring.md. Do NOT create judge.ts or debug.ts for a standard stdin/stdout problem — the judge server auto-generates both, and committed default-content copies are rejected by the validator. Write a custom judge.ts (with a matching debug.ts) only when the problem needs special judging (GUI, LLM, command-based, etc.); a genuinely custom debug.ts may also accompany a standard stdio problem on its own when the problem needs customized debugging.
  5. Create test inputs under test_cases/: at least 1 example_* case (shown to learners and mirrored as入力例/出力例in the statement; for an input-only problem the出力例shows the judged output file, see references/problem-format.md) and at least 1 hidden case, at least 4 in total. Make inputs diverse and include edge cases (minimum/maximum constraint values, boundary conditions such as 0 or negative numbers where allowed). All inputs must be distinct. A program that reads files instead of stdin takes them from <id>.fin/; a program that writes files is judged with <id>.fout/ (see references/problem-format.md).
  6. Produce every .out file and every file under .fout/ by RUNNING the model answer locally — never hand-write expected outputs. For a plain stdin/stdout problem:
Installs
196
First Seen
Sep 6, 2026