視覺模型看不出版面錯誤？改用像素量度

I generate the social cards for my projects as HTML and CSS, screenshot them with headless Chromium, and publish them without a person looking at each slide first. That last part is the reason I needed a gate: something has to decide whether a card is safe to publish, and it has to be cheap enough to run on every piece, every day.

The obvious candidate was vision. Screenshot the card, send it to a multimodal model, ask whether any text is cut, overlapping or missing. Multimodal models read text well and describe screenshots well, so this looks like a solved problem.

It did not survive contact with the pipeline. What follows are the failures, and the check that replaced them.

Asking a vision model conditions the answer

The first version asked, literally: "is any text cut off in this image?" The model said yes on pieces where nothing was cut. I measured those margins by hand and they were comfortable.

The same question fails in the other direction. When the layout really is broken, a leading question pushes the model toward a confident sentence with a location attached, and the location is often wrong. A model that produces plausible coordinates is worse than one that says it cannot tell, because the coordinates end up in a report and the report gets trusted.

What changed my mind was not a benchmark. It was a card that passed the vision check and shipped with the footer missing, phone number included. The measurement below caught the same class of defect on the next run, in two seconds, with the row number and no adjectives.

Ink is any pixel that differs from the background

The naive pixel check fails on a light theme, and it fails quietly. Define ink as "light pixels" and a cream background becomes ink: every row reports text at y=12, a batch of cards all fail, and nothing is actually wrong.

The rule that held up separates three things.

• Ink is any pixel whose color differs from the piece's background, sampled at a corner the design never uses for content.
• Decoration is separated from content by distance to that background: 0 to 24 is background, 25 to 49 is halo or glow, 50 and above is content.
• Stroke thickness separates a grid line from a glyph. A rule is one pixel wide. A letter never is.

from PIL import Image

img = Image.open(path).convert("RGB")
w, h = img.size
bg = img.getpixel((4, 4))

def distance(px):
    return max(abs(px[i] - bg[i]) for i in range(3))

def kind(px):
    d = distance(px)
    if d < 25:
        return "background"
    if d < 50:
        return "halo"
    return "content"

Those thresholds are not universal. They come from one palette, and they age the moment the palette changes. Recalibrate when the design changes, or the check keeps passing while measuring the wrong layer.

A cut is local, so measure every line

A card can be perfect everywhere except one line, which is exactly the case a whole-image average hides. Group the rows that contain content into bands, then take the leftmost and rightmost ink inside each band.

def bands(mask, h, gap=30):
    rows = [y for y in range(h) if any(mask[y])]
    out, current = [], []
    for y in rows:
        if current and y - current[-1] > gap:
            out.append(current)
            current = []
        current.append(y)
    if current:
        out.append(current)
    return out

For each band, min_x and max_x. If min_x < 6 or max_x > w - 6, the card fails and the report names the band. Six pixels is the tolerance. Below that, a glyph is touching the frame, and touching the frame on a phone means clipped.

Chromium's --screenshot does not capture a tall page

This one cost an hour and it is not a layout bug. A 1080 by 1350 card came out with a white strip at the bottom. Below y=1232 there was no ink at all, and the footer carrying the phone number, the handle and the site had vanished.

The DOM was correct. The CSS was correct. Every assertion against the DOM passed, because the DOM was never the problem. chromium --screenshot fits the page into the window and stretches the result, so the capture is fake exactly where the content sits lowest.

The fix is to drive Chromium over the DevTools protocol, set the real metrics, and ask for the region beyond the viewport:

cdp("Emulation.setDeviceMetricsOverride",
    width=1080, height=1350, deviceScaleFactor=1, mobile=False)
shot = cdp("Page.captureScreenshot", format="png", captureBeyondViewport=True)

Then measure the output by pixel. Not by eye, and not by asking a model.

The failure I did not expect: a test that passes for the wrong reason

The gate that checks reading order failed four times before it was right.

1. Fixed positions for the numbers. Broke the first time the layout moved.
2. Height only. Found two of the three numbers and called the order valid.
3. Height plus width, no control. Indistinguishable from a function that always returns sorted.
4. Height plus width with a control. The control glued the bands into one block because it used a 24 pixel gap while the grouping constant was 30, the order came out empty, and an empty order passed.

That fourth one is the useful story. The control existed, it ran, and it proved nothing, because the control has to produce the wrong answer for a wrong input. Both paths also have to call the same function. If the control has its own copy of the logic, you are testing the fixture.

Text in the frame has to be explicit data

The corners of each card carry the state, the badge and the slide number. All three passed their checks while the content was wrong, because the state was derived from the last word of the badge. A piece labelled "GOOGLE CLOUD" reported its state as "CLOUD", another reported "LADO", and no check could see it since the value was already wrong before it arrived.

Deriving one field from another saves a line of code and creates a class of bug the verifier cannot catch. Frame text is data. Write it as data.

What the gate looks like now

Every piece runs the same order: render, measure ink, check margins per line, check reading order, fingerprint the footer region. Measurement decides. A vision model is at most a second opinion, and when it is used, the prompt describes what to look at instead of asking whether something is broken.

The rule I took from all of this: if a check can only ever say yes, it is not a check. Feed it a card that is genuinely broken and watch it fail, because on the day it is wrong, nobody else is looking at the image either.

Se você está montando um pipeline parecido e prefere português, deixa um comentário que eu respondo em português. Esse texto saiu de um sistema que gera, verifica e publica arte de carrossel sem revisão humana, e a parte de verificação foi a que me ensinou mais.