Benchmark Fetcher Skill

Automate the fetching of benchmark performance data from leaderboard websites and update model manifests with the latest scores using advanced browser automation.

Overview

This skill extends benchmark data collection by automating visits to 6 major AI model leaderboard websites, extracting performance scores, and updating model manifests in manifests/models/ with the latest benchmark data.

Key Features:

• Automated Data Collection: Uses Playwright MCP to visit and extract data from 6 leaderboard websites
• Intelligent Model Mapping: Maps website model names to manifest IDs using configurable mappings
• Always Overwrite: Updates manifests with latest benchmark values
• Error Resilient: Retry logic with exponential backoff and graceful degradation
• Comprehensive Reporting: Detailed completion reports with unmapped models and update statistics

Supported Benchmarks