Multimodal Prompting

Disclaimer. Multimodal capabilities and limits (max image size, audio duration, frame extraction) vary by provider and model. The provider's documentation is authoritative.

The model can see images, hear audio, watch video. What it can't do is decide what matters — that's the prompt's job. Without guidance, a multimodal model describes colors, layout, irrelevant background; with guidance, it focuses on the conversion rate, the error indicator, the data anomaly you actually care about.

When to use

• Input is an image, screenshot, document scan, audio file, or video.
• Output is an image (generation prompts).
• Mixed input: text + image (debug a UI bug + screenshot).

Image analysis — the structured pattern

Don't ask "what do you see?" Ask for specific extractions:

You will be given a screenshot of <surface>. Extract the following.