Hive-to-Paimon Data Migration

Verified agent platforms: Claude / Qoder / OpenCode / Codex. Other platforms are not compatibility-tested.

⚠️ This skill performs production data writes and overwrites. Preview every write operation with --dry-run first, and execute only after human review. All output is for reference and must be verified by a human.

Safety Red Lines

1. Overwrites are irreversible: INSERT OVERWRITE clears all data in the target Paimon table; --force runs DROP TABLE then recreates. Before running, confirm the target table is empty, disposable, or already backed up.
2. Never commit credentials: OSS AK/SK, Spark password, and Metastore password in config.ini are highly sensitive. Never commit them to Git/SVN. Add config.ini to .gitignore and inject credentials via environment variables (see "Credential Security").
3. No real-prefix placeholders: Never write example AKs with real prefixes like LTAI.... Always use <YOUR_AK> / <YOUR_SK>.
4. Large-cluster protection: The default rclone concurrency of 64 may overwhelm the source HDFS or target OSS. On production clusters, evaluate with low concurrency first (e.g. --max-parallel 2 --transfers 16) before scaling up.
5. DLS direct-read boundary: --direct-read only applies to OSS-HDFS (DLS) sources. Enabling it on plain HDFS results in unreadable data.
6. Writes require confirmation: Before running CREATE TABLE / DROP TABLE / INSERT OVERWRITE / rclone copy, the agent must show the impact and obtain explicit confirmation (see references/agent-rules.md).

Tooling Overview

This skill depends on no external MCP tools; it is implemented via a Python CLI plus external services: