Real-Time Protocol Guidance Prompts

Overview

realtime_protocol_guidance_prompts is the prompt-generation layer of the LabOS real-time XR guidance stack. Given the current first-person video frame (or VLM-derived scene description) and the active protocol context — current step, expected action, detected deviation, and operator state — it produces a single short guidance string optimized for voice synthesis (TTS) or AR overlay display. The output is imperative, concise, and actionable: "Add 50 µL buffer now." / "Vortex before proceeding." / "Step 4 complete. Move to pipette." — enabling hands-free, eyes-on-bench guidance that keeps the researcher in flow without interrupting to read a screen.

When to Use This Skill

Use this skill when any of the following conditions are present:

• Live XR-assisted protocol execution: An operator wearing an XR headset is executing a wet-lab protocol and needs step-by-step voice or AR prompts at each transition — the agent must generate the next prompt based on current frame and protocol state.
• Deviation correction delivery: protocol_video_matching has detected a deviation (e.g., skipped step, wrong volume) and the agent must convert the deviation into a corrective prompt: "Vortex the lysate for 30 seconds before adding ethanol."
• Step confirmation and next-step cue: The current step has been validated (VLM detected expected action) and the agent must generate the next prompt: "Step 4 complete. Proceed to add 600 µL ethanol."
• Timeout or stall detection: No expected action has been observed for a configurable duration; the agent must generate a reminder or prompt: "Continue with step 5: add ethanol to the lysate."
• Hands-free operation: The researcher cannot look at a screen; guidance must be delivered via TTS or minimal AR text overlay — prompts must be short (≤ 15 words for TTS, ≤ 8 for AR overlay).
• Multi-language support: The same protocol context must produce prompts in different languages (English, Spanish, Mandarin) for international teams or training.
• Training mode: A trainee is learning a protocol; prompts include optional hints or warnings: "Careful: add ethanol slowly to avoid precipitation."
• Error recovery: The operator has acknowledged a correction or asked for help; the agent must generate a recovery prompt: "Resume from step 6: centrifuge at 12,000 × g for 2 minutes."