皮膚癌模型為何無法處理真實照片

A 10-year-old CNN can beat frontier LLMs at skin cancer diagnosis, but only when the input looks like the training data. In practice, that almost never happens.

Your model sees dermoscopic images: high-resolution, ring-light illumination, no hair or text. Your users send phone photos taken under bathroom LEDs, with flash glare, arm hair, and a ruler sticker in frame. The same architecture that scores 95% on ISIC benchmarks drops to near-random when the distribution shifts. This is not a model capacity problem. It is a data pipeline problem.

By the end of this piece you will be able to:

• Identify the three most common distribution shifts that break dermoscopic CNNs
• Build a preprocessing pipeline that normalizes real-world photos into model-ready inputs
• Choose between fine-tuning and prompt-based augmentation based on your data budget

The three shifts that kill your model

Dermoscopic datasets like ISIC are collected under controlled conditions. Real-world photos are not. Three shifts dominate the failure mode:

1. Illumination variance: Phone flash creates specular highlights that obscure lesion borders. Dermoscopic images use diffuse ring lighting.
2. Scale and framing: Users crop tightly or include irrelevant anatomy. Dermoscopic images are standardized to a fixed field of view.
3. Artifacts: Arm hair, text labels, compression noise, and skin texture from phone cameras introduce features the model never saw during training.

These are not edge cases. They are the default input.

Normalize before you classify

The fix is not a bigger model. It is a preprocessing pipeline that maps real photos into the distribution the model expects. Here is a minimal implementation using OpenCV and Pillow:

import cv2
import numpy as np
from PIL import Image

def preprocess_skin_photo(image_path, target_size=(224, 224)):
    img = cv2.imread(image_path)
    if img is None:
        raise ValueError(f"Could not read {image_path}")

    # Remove hair artifacts using inpainting
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (17, 17))
    blackhat = cv2.morphologyEx(gray, cv2.MORPH_BLACKHAT, kernel)
    _, mask = cv2.threshold(blackhat, 10, 255, cv2.THRESH_BINARY)
    cleaned = cv2.inpaint(img, mask.astype(np.uint8), 3, cv2.INPAINT_TELEA)

    # Normalize illumination using CLAHE on the luminance channel
    lab = cv2.cvtColor(cleaned, cv2.COLOR_BGR2LAB)
    l, a, b = cv2.split(lab)
    clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
    l = clahe.apply(l)
    lab = cv2.merge([l, a, b])
    normalized = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)

    # Resize and center crop to match dermoscopic aspect ratio
    pil_img = Image.fromarray(cv2.cvtColor(normalized, cv2.COLOR_BGR2RGB))
    pil_img = pil_img.resize((256, 256), Image.Resampling.LANCZOS)
    left = (256 - target_size[0]) // 2
    top = (256 - target_size[1]) // 2
    pil_img = pil_img.crop((left, top, left + target_size[0], top + target_size[1]))

    return np.array(pil_img) / 255.0

This pipeline removes hair, flattens lighting, and crops to the same aspect ratio the model was trained on. It is not perfect, but it closes the gap enough to make the model usable.

Fine-tune or augment?

If you have labeled real-world photos, fine-tune the final layers of your CNN. If you only have dermoscopic labels, use data augmentation to simulate real-world conditions during training.

For most teams, synthetic augmentation is the pragmatic starting point. Add random brightness, blur, hair overlays, and JPEG compression to your training loop:

import albumentations as A

def get_augmentation_pipeline():
    return A.Compose([
        A.RandomBrightnessContrast(brightness_limit=0.3, contrast_limit=0.2, p=0.7),
        A.GaussBlur(blur_limit=5, p=0.3),
        A.JpegCompression(quality_lower=40, p=0.5),
        A.CLAHE(p=0.3),
        A.CoarseDropout(width_percent=0.1, height_percent=0.1, p=0.2),
    ])

What still breaks

Even with preprocessing, two failure modes persist:

• Severe scale mismatch: If the user includes too much surrounding skin, the lesion occupies fewer than 10% of the frame. No amount of cropping recovers the detail the model needs.
• Pathology outside the training distribution: If the lesion type is rare in ISIC (e.g., amelanotic melanoma), the model will miss it regardless of preprocessing.

For these cases, fall back to a human triage queue. Flag low-confidence predictions and route them to a dermatologist.

Key takeaways

• Real-world phone photos differ from dermoscopic training data in illumination, scale, and artifacts -- not in lesion biology.
• A preprocessing pipeline that removes hair, normalizes lighting, and crops to the right aspect ratio closes most of the performance gap.
• Fine-tune on real photos if you have them; otherwise, augment dermoscopic images with realistic artifacts during training.
• Always include a human fallback for low-confidence predictions and rare lesion types.

Source

A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis -- I added a working preprocessing pipeline, an augmentation strategy, and a failure-mode analysis that the source does not cover.

Support this work

These write-ups are researched and published with no paywall, sponsor, or tracking. If one saved you an afternoon, a small tip keeps them coming.

USDT, USDC or USDD · TRC-20 (Tron)

TFTNsfyomKrnUutRjBTGVULp19ByW29KbY