extract_experiment_data_from_video
Extract Experiment Data from Video
Overview
extract_experiment_data_from_video is the universal sensor layer of the LabOS video-to-data pipeline: it turns unstructured lab footage from XR headsets (Meta Quest, HoloLens 2) or fixed bench cameras into typed, timestamped measurement tables. The skill applies a configurable battery of computer-vision extractors — liquid-level tracking, colorimetry, turbidimetry, object counting, OCR-based instrument reading, and spectral intensity profiling — and emits a structured time-series JSON or flat CSV that any downstream analysis, charting, or ELN-integration skill can consume directly, bridging the gap between physical bench observations and digital data without manual transcription.
When to Use This Skill
Use this skill when any of the following conditions are present:
- Colorimetric or turbidity assays: A reaction vessel, cuvette, or multi-well plate is changing color or becoming cloudy over time (Bradford assay, OD600 bacterial growth, BCA protein assay, enzymatic color reactions, precipitation) and the change must be quantified from video rather than a plate reader.
- Liquid volume monitoring: The fill level in a tube, flask, or reservoir is changing during an experiment (evaporation tracking, titration endpoint detection, droplet dispensing QC) and must be logged as a volume timeseries.
- Pipette and dispensing readout: The operator's pipette volume dial or digital display appears in the XR field of view and the dispensed volumes must be recorded as part of a protocol audit trail (used together with
protocol_video_matching). - Instrument display capture: Lab instruments (balances, pH meters, thermometers, centrifuge RPM displays, PCR machine screens, incubator panels) are in frame and their readings must be extracted and logged without manual note-taking.
- Cell, colony, or spot counting: Petri dishes, hemocytometers, colony plates, or gel images appear in the video and require automated counting at one or more time points.
- Gel electrophoresis quantification: An agarose or SDS-PAGE gel is being imaged or appears in the XR field of view; band migration distances and pixel intensities must be extracted for downstream MW estimation or densitometry.
- Multi-step reaction monitoring: A benchtop chemical or biological reaction produces observable visual changes across multiple steps (color transitions, precipitate formation, foam, phase separation) that serve as proxy indicators for reaction progress.
- Passive ambient logging: A fixed camera is recording a bench area over hours or days; the skill extracts any detectable change events (object appearance/disappearance, color shifts, liquid level changes) as a sparse event log for later review.