跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 4 天前AI 評分53

Squoosh Bench 評估 UX-1 在 318 項歷史網站 A/B 測試中的表現

Squoosh Bench: Evaluating UX-1 on 318 Historical Website A/B Tests

AI 導讀

Squoosh Bench 評估 UX-1 對歷史網站 A/B 測試結果的預測,在 318 項測試中正確判定 267 項,準確率為 84.0%。UX-1 以人類瀏覽數據微調,每個測試組模擬 1,000 名購物者;Qwen3 Next 80B 正確判定 196/318 項,準確率為 61.6%。UX-1 在 126 項測試中預測變體勝出,其中 118 項實際勝出、4 項落敗,另有 4 項結果不明確。

正文

Published on Nov 18

Authors:

,

,

,

Abstract

Experimentation teams must decide which proposed website changes to test with live traffic. We evaluated whether forecasts from simulated shoppers could inform that choice. We screened 412 historical website A/B tests and evaluated four systems on the 318 that ran longer than 21 days. Each system predicted an original win, a variant win, or an inconclusive result. Page roles were masked and recorded outcomes were withheld during inference. Squoosh UX-1, a model fine-tuned on human browsing data, simulated 1,000 shoppers per arm and correctly classified 267 of 318 outcomes (84.0%, 95% Wilson interval 79.5-87.6%, treating experiments as independent). Qwen3 Next 80B, the strongest general-purpose configuration on outcome accuracy, correctly classified 196 of 318 (61.6%). UX-1 favored the variant in 126 experiments. In the live tests, 118 of those variants won (93.7%), four lost, and four had inconclusive results. The remaining 192 experiments included 19 additional variant wins.

View arXiv page View PDF Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.25200 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.25200 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.25200 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co