---
id: con-multimodal
schema: concept/1
layer: garden
status: established
name: Multimodal AI
aka: ["vision", "image input", "voice mode", "talk to AI", "image generation"]
summary: >
  AI that can take in or produce more than text, such as photos, screenshots, PDFs, voice, audio
  and video, so you can show it the problem as well as describe it.
first_seen: "2023"
relations:
  builds_on: [con-large-language-model]
  evolved_from: []
  related: [con-artifacts]
  parallels:
    - idea: "sending a photo of the problem instead of describing it"
      field: "customer support"
      note: "Support teams learned long ago that a picture removes a round of back-and-forth."
implementations:
  - { provider: openai, name: "ChatGPT voice and image input" }
  - { provider: google, name: "Gemini Live" }
  - { provider: google, name: "NotebookLM Audio Overviews" }
  - { provider: anthropic, name: "Claude image and PDF understanding" }
competencies: [cmp-describe-the-task]
origin:
  kind: ai-proposed
  by: "seed agent 2026-10-16"
  rationale: "Showing is often easier than writing for nontechnical users; widens what counts as a prompt."
---

Multimodal means the AI works across different kinds of material. You can upload a photo of a whiteboard and ask for the action items, share a screenshot of a confusing error, hand over a scanned form, talk through an idea out loud while walking, or ask for an image or an audio summary back.

For many people this is the easiest way in. Instead of carefully writing out what a chart shows, you show the chart. Instead of typing meeting notes, you speak them. The AI still needs to know what you want from the material, so a short instruction alongside the picture or recording still matters.

The same cautions apply as with text. AI can misread a blurry number in a photo or mishear a name, and it will report the mistake just as confidently.

## Example

A facilities manager photographs a cluttered storage room and asks for a labelled layout plan with what to keep, move and recycle. She corrects two items it misidentified, then prints the plan for the team.

## How it connects

- It extends **prompting**: the prompt can be a picture, a file or your voice.
- Outputs like diagrams and pages connect to **artifacts**.
- Misread images are a source of **hallucination**, so **verification** still applies.

## Try it

Photograph something messy, like your desk or a whiteboard, and write down three things you expect the AI to pull out of the picture. Upload it, ask, and compare its list with yours.