Can you use AI to analyse images?

Short answer: it’s a good reader and a poor judge. Upload a photo, screenshot, chart or diagram, ask a question, and you get a plain-language answer. It’s useful for an error message, a confusing settings page, a flowchart someone else drew or a graph in a report. It goes wrong on fine detail, and it sounds just as sure of itself when it’s wrong.

The details below are for ChatGPT, because OpenAI publishes what its image feature can and can’t do. Other AI tools have their own rules, so check their help pages.

Analysing an image is a different job from changing it. If you want a better-looking picture, see editing photos with AI or upscaling images. Here, the AI reads what’s already there and leaves your file alone.

What it handles and where it slips

OpenAI says image inputs work on Free and paid plans, subject to plan-specific usage limits, on the web and in the iOS and Android apps. Every ChatGPT model can accept images. You can ask about objects, analyse documents and add more images later in the same chat.

OpenAI’s own list of limitations is worth reading before you rely on an answer:

• Small text: enlarge it to help it read, without cropping out important detail.
• Rotated images: it may misread rotated or upside-down text and images.
• Non-Latin text: it does worse with alphabets such as Japanese or Korean.
• Charts: it may struggle where colours or line styles (solid, dashed, dotted) vary.
• Position: it struggles with tasks that need precise positions, such as reading a chess board.
• Counting: it may give only approximate counts.
• Odd shapes: panoramic and fisheye images give it trouble.
• Plain mistakes: it may describe or caption a picture wrongly.

It also doesn’t read the original file name or metadata, and it resizes images before it looks at them.

Then there’s medical images. OpenAI says the model is not suitable for interpreting specialised medical images like CT scans and shouldn’t be used for medical advice. Its 2023 system card for GPT-4V found it interpreted medical images inconsistently and said OpenAI did not consider it fit for any medical function. Apply the same caution to a rash, a mole or anything where a wrong answer could hurt someone, and ask a professional.

How to do it well

Uploading is quick. Tap the + in the prompt area and choose Add photos & files, or drag the image into the text box, or paste it from your clipboard.

The question matters more than the upload:

1. Say what it is and what you want. “This is a screenshot of my bank’s login page. Explain what the error message means” beats “what is this?”
2. Point at the part you care about. OpenAI suggests marking up the image with a photo editing tool before you upload it, so ChatGPT focuses on that area.
3. Crop and enlarge. If the text is small, upload a close-up instead of the whole page.
4. Ask one thing at a time, then check every name, number and date against the image yourself.

For long or multi-page documents, a PDF route usually works better than a pile of screenshots; see summarising a PDF with AI. And if the image is a worksheet or exam question, use the answer to understand the work, never to hand it in. OpenAI’s usage policies ban academic dishonesty, and using AI on homework covers where the line sits.

When it won’t take your image

When an upload fails, the cause is usually one of the published limits:

• Format: ChatGPT accepts PNG, JPEG (.jpeg and .jpg) and non-animated GIF. Convert anything else to PNG or JPEG.
• Size: each image can be up to 20MB.
• Quantity: OpenAI doesn’t publish a fixed number of images per chat. It depends on image size and how much text goes with them, so if you hit problems, use fewer or smaller images.
• Plan limits: usage limits vary by plan, so you may run out for a while.

A refusal is a different thing. If you ask it to name the person in a photo, expect a no. That’s policy, not a fault (see below).

Rules and risks

Photos of other people. UK GDPR doesn’t apply to processing personal data “in the course of a purely personal or household activity”. The ICO explains this means personal use with no link to work or business, such as taking pictures for your own enjoyment. Asking ChatGPT to describe a family photo for your own interest sits there. Do the same for a client, an employer or a business and that exemption no longer covers you, so read the ICO guidance first.

Children. The ICO’s guidance for organisations says children merit specific protection because they may be less aware than adults of the risks involved. Even in personal use, think twice before uploading photos of other people’s children.

Identifying people. OpenAI’s 2023 system card said it had trained the model to refuse requests to identify people in photos, from public figures to private individuals. Its usage policies ban building facial recognition databases without consent, real-time biometric identification in public spaces and attempts to profile or monitor people without their authorisation.

Location. The same system card said the model can often work out the city a photo was taken in, which it flagged as a privacy concern for people who don’t want their location known. It said the model generally didn’t go deeper than that. Before you upload, look at what the background gives away: street signs, house numbers, a school badge.

Documents and training. OpenAI treats images the same way as the rest of your chats. In ChatGPT, open Settings, then Data controls, and turn off Improve the model for everyone so new conversations aren’t used to train its models. A Temporary chat isn’t used for training either, though OpenAI may keep it for up to 30 days for safety purposes. Even with training off, giving a thumbs up or down can mean that whole conversation is used for training. Whatever your settings, keep passports, bank statements and anything with account numbers out of the chat.

This is general information, not legal or medical advice.