pig-expert
Installation
SKILL.md
Apache Pig Expert
Based on "Programming Pig" by Daniel Dai (O'Reilly), with modern focus on AWS EMR and GCP Dataproc execution.
Note on currency: Pig 0.17+ is the stable release. AWS EMR and GCP Dataproc both support Pig as a managed component. While Pig Latin itself is stable, cloud integration patterns (S3/GCS paths, cluster configs, IAM) are the primary modern concern.
Quick Reference — Load the Right File
| Task | Reference File |
|---|---|
| Pig Latin syntax, data types, relations, bags, tuples, maps | pig-latin-basics.md |
| FOREACH, FILTER, GROUP, JOIN, COGROUP, ORDER, DISTINCT, LIMIT | relational-operators.md |
| Nested FOREACH, replicated join, skewed join, merge join, FLATTEN | advanced-operators.md |
| LOAD/STORE with PigStorage, ORC, Parquet, Avro, JSON, HCatalog/Hive Metastore | input-output.md |
| Writing UDFs in Python, Java, or JavaScript; Eval/Filter/Aggregate/LoadFunc | udfs.md |
| Running Pig on AWS EMR — cluster launch, S3 paths, steps, IAM, logging | aws-emr.md |
| Running Pig on GCP Dataproc — cluster launch, GCS paths, jobs, IAM, logging | gcp-dataproc.md |
| ILLUSTRATE, DESCRIBE, EXPLAIN, job counters, profiling, multi-query optimization | diagnostics-performance.md |