6.42 · Image support — Claude đọc ảnh (Vision)

Mở đầu: Claude "nhìn thấy"

• Describe image
• Count objects
• Extract text (OCR)
• Analyze diagram / chart
• Compare 2 images
• Identify object, scene, style
• Medical imaging (với disclaimer)
• Satellite / aerial analysis

Limits

Example: 1000×1000 image = ~1333 tokens. Budget accordingly.

Sending image — Base64

import base64
from anthropic import Anthropic

client = Anthropic()

with open("image.png", "rb") as f:
    image_bytes = base64.standard_b64encode(f.read()).decode("utf-8")

messages = [{
    "role": "user",
    "content": [
        {
            "type": "image",
            "source": {
                "type": "base64",
                "media_type": "image/png",
                "data": image_bytes
            }
        },
        {
            "type": "text",
            "text": "What do you see in this image?"
        }
    ]
}]

response = client.messages.create(
    model="claude-sonnet-4-6-20260205",
    max_tokens=1000,
    messages=messages
)

print(response.content[0].text)

Sending image — URL

messages = [{
    "role": "user",
    "content": [
        {
            "type": "image",
            "source": {
                "type": "url",
                "url": "https://example.com/image.jpg"
            }
        },
        {"type": "text", "text": "Describe this."}
    ]
}]

URL must be publicly accessible. Anthropic fetch server-side.

Multiple images

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "source": {...img1}},
        {"type": "image", "source": {...img2}},
        {"type": "text", "text": "Compare these two images."}
    ]
}]

Up to 100 images per request. Useful cho:

• Before/after comparison
• Product variants
• Batch analysis

Prompting cho image

Same rules as text (Module 3):

• Clear & direct
• Specific
• Examples
• XML structure

Simple prompt — poor results

text = "How many marbles?"

Claude count có thể sai với dense image.

Structured prompt — great results

text = """Count marbles in the image using this methodology:

1. Identify each unique marble one at a time, numbering as you go
2. Verify by counting differently: bottom-left to top-right, row by row

What is the exact count?"""

Accuracy tăng từ 60% → 95% cho counting task.

Few-shot với image

Dùng image làm example:

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "source": {...reference_image}},
        {"type": "text", "text": "This image has 12 marbles. The pattern is 3x4 grid."},
        {"type": "image", "source": {...target_image}},
        {"type": "text", "text": "How many marbles in this second image?"}
    ]
}]

Claude học pattern từ reference → apply to target.

Case study: Fire risk assessment

Scenario

Insurance company wants automated fire risk rating cho properties từ satellite images.

Naive prompt

"Rate fire risk 1-10"

Output: inconsistent, subjective.

Production prompt

prompt = """Analyze satellite image of property with these steps:

1. **Residence identification**: Locate primary residence:
   - Largest roofed structure
   - Typical residential features (driveway, geometry)

2. **Tree overhang analysis**: For trees near residence:
   - Identify branches over roof
   - Estimate % roof coverage (0-25%, 25-50%, 50-75%, 75%+)
   - Note density

3. **Fire risk factors**:
   - Ember catch points
   - Fuel paths from wildland to structure
   - Proximity to chimneys/vents

4. **Defensible space**:
   - Continuous canopy?
   - Fuel ladders (ground → tree → roof)?

5. **Final rating** (1-4):
   - 1 (Low): no overhang, good defensible space
   - 2 (Moderate): <25% overhang, some separation
   - 3 (High): 25-50% overhang, connected canopies
   - 4 (Severe): >50% overhang, dense vegetation

For each item 1-5, write ONE sentence finding. End with rating number.
"""

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "source": {...satellite_image}},
        {"type": "text", "text": prompt}
    ]
}]

Output

1. Residence: Single-story ranch home, ~2000 sqft, central.
2. Tree overhang: 5 trees within 10m, 30% roof coverage from SE oak.
3. Fire risk: Eaves exposed, branch touches roof (bridge point).
4. Defensible space: Trees form continuous canopy N→W of house.
5. Rating: 3 (High Risk)

Consistent, auditable, scalable. 1000 properties/day automated.

Case studies theo ngành

Healthcare — X-ray second opinion

Prompt: "Identify abnormalities. Rate severity. Flag for radiologist review."

Disclaimer: Not diagnostic. Assists human radiologist.

Manufacturing — Defect detection

Factory camera + Claude vision → detect visual defects on assembly line.

Retail — Planogram compliance

Store shelf photos → Claude verify products placed correctly per planogram.

Legal — Document OCR + analysis

Scanned contract → extract clauses, flag issues. Better than traditional OCR for handwritten / poor scan.

Design — Style match

Brand guidelines image + new creative asset → "Does this match brand style?"

Optimization

Resize before sending

from PIL import Image

img = Image.open("large.jpg")
img.thumbnail((1024, 1024))  # resize keeping aspect ratio
img.save("resized.jpg", quality=85)

Smaller image = fewer tokens = cheaper. Balance quality.

Cost estimation

1000×1000 image = ~1333 input tokens

Cost Sonnet: 1333 × $3 / 1M = $0.004 per image

Processing 10,000 images/day = $40/day. Reasonable.

Cache common context

Nếu prompt instructions long + reused → enable prompt caching (bài 6.47-6.49) → 90% discount.

Anti-patterns

Giant images

5MB raw → slow upload, expensive tokens.

Fix: Resize before. 1024px usually enough.

Low-res cho detail work

256×256 cho OCR → text illegible.

Fix: Use appropriate resolution for task.

Prompt "what's this?" generic

Claude describe generally, không specific.

Fix: Guide với methodology (steps).

Trust numerical count blindly

Claude có thể undercount dense objects (> 20).

Fix: Use step-by-step prompt + verify pattern.

Áp dụng ngay

Bài tập 1: Describe 3 images (20 phút)

3 image khác nhau (personal photos). Prompt:

1. Simple: "What's in this image?"
2. Structured: Methodology-based
3. Extract: "List all objects with counts"

Compare output quality.

Bài tập 2: Chart analysis (15 phút)

Screenshot chart từ web. Ask Claude:

• Extract data points
• Identify trend
• Suggest 3 insights

Test với structured prompt.

Tóm tắt bài học