timemachine-bench-benchmark-evaluating-capabilitie

Installation
SKILL.md

Dependency Migration Agent: Systematic Python Library Upgrade Workflow

This skill enables Claude to perform repository-level dependency migration — adapting Python codebases when dependency updates cause test failures. Based on the TimeMachine-bench methodology (Fujii et al., EACL 2026), it implements a disciplined agent loop: run tests, analyze failures, apply minimal targeted edits, re-test, and revert when wrong. The key insight from the paper is that naive LLM-driven migration produces two failure modes — spurious solutions (changes that pass tests by exploiting low coverage rather than fixing the real issue) and unnecessary edits (over-broad changes from poor tool-use strategy). This skill teaches the disciplined approach that avoids both.

When to Use

  • When the user reports test failures after upgrading a Python dependency (e.g., pytest, numpy, pandas, django, pydantic)
  • When the user asks to migrate a codebase from one library version to another (e.g., "upgrade from Pydantic v1 to v2")
  • When pip install --upgrade or a lockfile update breaks existing tests
  • When the user needs to adapt code to breaking API changes in a new library release
  • When performing a planned major version bump of a dependency across a project
  • When investigating which code changes are required by a dependency changelog

Key Technique

The TimeMachine-bench paper establishes that software migration is fundamentally a test-driven, iterative repair task. The benchmark constructs real-world migration problems by finding GitHub repositories where test suites pass under old dependencies but fail under new ones. The agent's job: modify source code (never test code) until tests pass again under the new dependency versions.

The agent operates with a constrained tool set: file navigation (list_dir, search_dir, search_file, view_file), surgical editing (edit_file with line ranges, replace_all_in_file for bulk renames), revert capability (revert_last), and containerized test execution (execute_tests). The critical constraint is minimality: only change what the dependency update broke. The paper's precision metric (prec@1) measures whether edits align with the gold-standard fix locations. High pass rate with low precision signals a spurious solution — the code passes tests but via incorrect changes.

Installs
1
GitHub Stars
6
First Seen
Jun 20, 2026
timemachine-bench-benchmark-evaluating-capabilitie — ndpvt-web/arxiv-claude-skills