phoenix-evals

Malicious tool definition detected

Tool: SKILL.md Description: --- name: phoenix-evals description: Build and run evaluators for AI/LLM applications using Phoenix.

Malicious tool definition detected

Tool: rules/axial-coding.md Description: # Axial Coding Group open-ended notes into structured failure taxonomies.

Malicious tool definition detected

Tool: rules/common-mistakes-python.md Description: # Common Mistakes (Python) Patterns that LLMs frequently generate incorrectly from training data. ## Legacy Model Classes ```python # WRONG from phoenix.evals import OpenAIModel, AnthropicModel model = OpenAIModel(model="gpt-4") # RIGHT from phoenix.evals import LLM llm = LLM(provider="openai", model="gpt-4o") ``` **Why**: `OpenAIModel`, `AnthropicModel`, etc.

Malicious tool definition detected

Tool: rules/error-analysis-multi-turn.md Description: # Error Analysis: Multi-Turn Conversations Debugging complex multi-turn conversation traces.

Malicious tool definition detected

Tool: rules/error-analysis.md Description: # Error Analysis Review traces to discover failure modes before building evaluators.

Malicious tool definition detected

Tool: rules/evaluate-dataframe-python.md Description: # Batch Evaluation with evaluate_dataframe (Python) Run evaluators across a DataFrame.

Malicious tool definition detected

Tool: rules/evaluators-code-python.md Description: # Evaluators: Code Evaluators in Python Deterministic evaluators without LLM.

Malicious tool definition detected

Tool: rules/evaluators-code-typescript.md Description: # Evaluators: Code Evaluators in TypeScript Deterministic evaluators without LLM.

Malicious tool definition detected

Tool: rules/evaluators-custom-templates.md Description: # Evaluators: Custom Templates Design LLM judge prompts.

Malicious tool definition detected

Tool: rules/evaluators-llm-python.md Description: # Evaluators: LLM Evaluators in Python LLM evaluators use a language model to judge outputs.

Malicious tool definition detected

Tool: rules/evaluators-llm-typescript.md Description: # Evaluators: LLM Evaluators in TypeScript LLM evaluators use a language model to judge outputs.

Malicious tool definition detected

Tool: rules/evaluators-overview.md Description: # Evaluators: Overview When and how to build automated evaluators.

Malicious tool definition detected

Tool: rules/evaluators-pre-built.md Description: # Evaluators: Pre-Built Use for exploration only.

Malicious tool definition detected

Tool: rules/evaluators-rag.md Description: # Evaluators: RAG Systems RAG has two distinct components requiring different evaluation approaches.

Malicious tool definition detected

Tool: rules/experiments-datasets-python.md Description: # Experiments: Datasets in Python Creating and managing evaluation datasets.

Malicious tool definition detected

Tool: rules/experiments-datasets-typescript.md Description: # Experiments: Datasets in TypeScript Creating and managing evaluation datasets.

Malicious tool definition detected

Tool: rules/experiments-overview.md Description: # Experiments: Overview Systematic testing of AI systems with datasets, tasks, and evaluators.

Malicious tool definition detected

Tool: rules/experiments-running-python.md Description: # Experiments: Running Experiments in Python Execute experiments with `run_experiment`.

Malicious tool definition detected

Tool: rules/experiments-running-typescript.md Description: # Experiments: Running Experiments in TypeScript Execute experiments with `runExperiment`.

Malicious tool definition detected

Tool: rules/experiments-synthetic-python.md Description: # Experiments: Generating Synthetic Test Data Creating diverse, targeted test data for evaluation.

Malicious tool definition detected

Tool: rules/experiments-synthetic-typescript.md Description: # Experiments: Generating Synthetic Test Data (TypeScript) Creating diverse, targeted test data for evaluation.

Malicious tool definition detected

Tool: rules/fundamentals-anti-patterns.md Description: # Anti-Patterns Common mistakes and fixes.

Malicious tool definition detected

Tool: rules/fundamentals-model-selection.md Description: # Model Selection Error analysis first, model changes last.

Malicious tool definition detected

Tool: rules/fundamentals.md Description: # Fundamentals Application-specific tests for AI systems.

Malicious tool definition detected

Tool: rules/observe-sampling-python.md Description: # Observe: Sampling Strategies How to efficiently sample production traces for review.

Malicious tool definition detected

Tool: rules/observe-sampling-typescript.md Description: # Observe: Sampling Strategies (TypeScript) How to efficiently sample production traces for review.

Malicious tool definition detected

Tool: rules/observe-tracing-setup.md Description: # Observe: Tracing Setup Configure tracing to capture data for evaluation.

Malicious tool definition detected

Tool: rules/production-continuous.md Description: # Production: Continuous Evaluation Capability vs regression evals and the ongoing feedback loop.

Malicious tool definition detected

Tool: rules/production-guardrails.md Description: # Production: Guardrails vs Evaluators Guardrails block in real-time.

Malicious tool definition detected

Tool: rules/production-overview.md Description: # Production: Overview CI/CD evals vs production monitoring - complementary approaches.

Malicious tool definition detected

Tool: rules/setup-python.md Description: # Setup: Python Packages required for Phoenix evals and experiments.

Malicious tool definition detected

Tool: rules/setup-typescript.md Description: # Setup: TypeScript Packages required for Phoenix evals and experiments.

Malicious tool definition detected

Tool: rules/validation-evaluators-python.md Description: # Validating Evaluators (Python) Validate LLM evaluators against human-labeled examples.

Malicious tool definition detected

Tool: rules/validation-evaluators-typescript.md Description: # Validating Evaluators (TypeScript) Validate an LLM evaluator against human-labeled examples before deploying it.

Malicious tool definition detected

Tool: rules/validation.md Description: # Validation Validate LLM judges against human labels before deploying.