data-engineering
Data Engineering
Build scalable data pipelines and infrastructure for big data processing.
Quick Start with Apache Spark
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, avg, sum, count
# Initialize Spark
spark = SparkSession.builder \
.appName("DataProcessing") \
.config("spark.executor.memory", "4g") \
.getOrCreate()
# Read data
df = spark.read.parquet("s3://bucket/data/")
More from pluginagentmarketplace/custom-plugin-ai-data-scientist
data-visualization
EDA, dashboards, Matplotlib, Seaborn, Plotly, and BI tools. Use for creating visualizations, exploratory analysis, or dashboards.
68statistical-analysis
Probability, distributions, hypothesis testing, and statistical inference. Use for A/B testing, experimental design, or statistical validation.
47reinforcement-learning
Q-learning, DQN, PPO, A3C, policy gradient methods, multi-agent systems, and Gym environments. Use for training agents, game AI, robotics, or decision-making systems.
38machine-learning
Supervised/unsupervised learning, model selection, evaluation, and scikit-learn. Use for building classification, regression, or clustering models.
33model-optimization
Quantization, pruning, AutoML, hyperparameter tuning, and performance optimization. Use for improving model performance, reducing size, or automated ML.
31computer-vision
Image processing, object detection, segmentation, and vision models. Use for image classification, object detection, or visual analysis tasks.
29