egocentric_view_to_structured_log
Egocentric View to Structured Log
Overview
egocentric_view_to_structured_log transforms raw first-person XR headset footage into a machine-readable experiment timeline. It processes the egocentric video stream frame-by-frame (or at configurable intervals), applies VLM or action-recognition models to infer what the operator did — pipetting, vortexing, adding reagent, loading centrifuge, labeling tube — and emits a structured log with timestamp, action type, object(s) involved, spatial location, and optional result or observation. The output is Markdown (human-readable timeline) or JSON (for programmatic consumption), suitable for ELN attachment, protocol compliance cross-reference, generate_scientific_method_section input, or audit trail documentation in the LabOS "from video to paper" pipeline.
When to Use This Skill
Use this skill when any of the following conditions are present:
- Experiment timeline documentation: A researcher needs a chronological record of what was done during an experiment — "at 14:23, added buffer to tube A1; at 14:25, vortexed; at 14:30, loaded centrifuge" — without manual note-taking.
- ELN or Benchling attachment: An electronic lab notebook entry requires an attached experiment log; the skill produces a Markdown or JSON file suitable for upload.
- Protocol compliance cross-reference: The structured log serves as ground truth for
protocol_video_matching— compare log events against protocol steps to detect deviations. - Methods section provenance:
generate_scientific_method_sectionconsumes the log to document the exact sequence of actions performed, with timestamps and objects. - Post-hoc experiment reconstruction: An experiment failed or produced unexpected results; the log enables step-by-step review to identify potential causes (e.g., "reagent added at 14:23, but protocol says add at 14:20 — 3 min delay").
- Training and assessment: A trainee's run is logged; the timeline is reviewed by a supervisor for feedback on sequence, timing, and technique.
- Audit trail for GLP/GMP: Regulated workflows require a timestamped record of every action; the log provides a structured, tamper-evident audit trail (when combined with video hash).
- Multi-operator coordination: When multiple people work at the same bench, the log can be tagged by operator (if face/ID available) or left anonymous for aggregate timeline.