Grad 和 Conn 能捕捉 alpha matte 中哪些 SAD 遺漏的問題，又有哪些捕捉不到

When I score an alpha matte, SAD is the column I read first. This time I scored eight mattes against the same 90-frame ground truth and put SAD next to two other columns, Grad and Conn. Grad told me something SAD flattened out. Conn, on this data, mostly didn't, which was useful to learn too.

Foreground and alpha came from a public-domain green-screen clip by VOA Learning English on Wikimedia Commons, 90 frames at 540×960 with a hand wave in the middle. I composited it onto a flat generated green and onto a busy generated background with I = αF + (1−α)B, so every frame has a known alpha. Known is not the same as correct, though. The "true" alpha is the output of a larger matting model (RVM ResNet50), not a hand-refined matte, and the AI method under test is from the same model family, so its scores may lean optimistic.

For the AI matte I used ImgIng https://imging.ai/ in its compatibility mode and pulled the alpha out of its transparent WebM export, once per background. The other six mattes are three FFmpeg chromakey settings, one hard green threshold and two controls made by altering the true alpha. Numbers below are 90-frame averages in the edge band (the true alpha's soft region plus about 7 px each side, 15.78% of the frame). They only mean something next to each other, not next to published results.

SAD scores a hard edge and a soft edge the same

SAD adds up |predicted − true| over the region. A pixel that is off by 0.5 costs the same whether it sits on a crisp step or inside a feathered fringe.

Chromakey at similarity 0.10 and blend 0, whose output is pure 0/1, scored 3.88. Control A, the true alpha blurred with a fixed σ=2 Gaussian, scored 3.91. A tie, with one edge too hard and the other too soft.

Grad compares the gradient magnitude of the two mattes after a Gaussian derivative filter, so it reacts to the shape of the edge. The hard-edged methods sit together, the threshold at 1.19 and chromakey 0.10 at 1.04, while the AI matte, which keeps a soft fringe, is at 0.53. SAD put the AI matte about a quarter below chromakey 0.10 (2.89 vs 3.88). Grad put it at roughly half.

Grad doesn't say which way the edge is wrong, though. Blur A reads 0.89 and chromakey 1.04, and both only mean "edge shape is off". Which way it's off takes a look at a crop. It also sees one frame at a time. Control B is the true alpha with fresh random noise (σ=0.06) added inside the edge band every frame. Its SAD is 1.53 and its Grad 0.03, the best in the table. Its flicker on pixels that should stay still was 11.844 (×1000), one of the two worst I measured.

Conn mostly moved with SAD here

Conn, the connectivity error from Rhemann et al. 2009, is meant to catch a different failure: foreground that has come loose from the main body, like background islands or broken hair. It sweeps a threshold and charges pixels that drop out of the largest connected region early. I followed the choices in the public Deep Image Matting evaluation code (0.1 steps, 0.15 cutoff); nobody has compared these numbers with the original Matlab version.

On the real methods Conn stayed close to SAD every time. Chromakey 0.10 was 3.60 against 3.88, the AI matte 2.67 against 2.89, chromakey 0.25 40.53 against 40.65. The busy-background AI run was 6.80 against 6.97, and that is the run where Conn should have broken away, since its error map for frame 13 shows dark fragments outside the person's outline. Across the six real runs Conn put them in the same order as SAD. The controls are where it moved: control A went from 3.91 to 3.50 and slipped below chromakey 0.10, and control B went from 1.53 to 0.79.

My guess, and it is only a guess: when most errors are flips between near 0 and near 1, a pixel wrong in value is also wrong in connectivity, so the two sums travel together. Conn would probably matter more on faint detached haze, and I had no case built for that.

Region matters for both. The whole-frame Conn I have for two methods barely moved from the band value (3.6008 vs 3.6003, 2.6724 vs 2.6718), but islands far from the outline sit outside the band by definition. SAD shows that trap more sharply: chromakey 0.25 is 93.37 over the full frame and 40.65 in the band, because that setting also removed dark garments and footwear as solid blocks.

I read SAD with Grad and Conn in one row

SAD is how much is wrong. Grad high relative to SAD means the edge profile is off, and a crop tells me which direction. Conn well below SAD means the errors are small and still attached to the subject. None of them looks across frames, so a separate flicker number sits next to them.

Source clip: "Filming with the green screen", VOA Learning English, Wikimedia Commons, public domain; cut and composited, modified. Backgrounds are generated.