Google MediaPipe

Overview

MediaPipe is Google's open-source framework for building on-device machine learning pipelines. It provides cross-platform APIs for vision, text, audio, and LLM inference tasks, plus a low-level graph-based pipeline framework for custom ML workloads.

Covers

• Computer vision tasks (face detection, face mesh, hand tracking, pose estimation, holistic landmarks, object detection, image classification/segmentation, gesture recognition)
• Text tasks (text classification, text embedding, language detection)
• Audio classification, (4) On-device LLM inference with MediaPipe GenAI/Tasks
• Model customization with MediaPipe Model Maker
• Visualizing/building ML pipelines with the MediaPipe Framework (graphs, calculators, packets)
• Drawing/rendering landmarks and detection results onto images/video
• Understanding MediaPipe's architecture and component ecosystem (Solutions, Tasks, Model Maker, Studio, Framework)

Architecture

MediaPipe has two layers: