跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-15AI 評分43

稀疏自編碼器中的表徵重新分配

Representation Redistribution in Sparse Autoencoders

AI 導讀

研究提出重新分配敏感度分數(RSS),用以識別稀疏自編碼器中反覆出現、方向一致的特徵變化,並量化其對下游讀出的貢獻,而不把表徵變化等同於失敗。研究評估了涵蓋八個語言模型的 27 個預訓練 SAE,發現提示詞變換在未用於篩選的變體上產生可重現的重新分配模式。

正文

Published on Oct 15

Authors:

Abstract

Sparse autoencoders decompose language model activations into sparse features, and the redistribution sensitivity score identifies recurring feature changes that contribute to downstream readout failures without equating representation change with failure.

Sparse autoencoders (SAEs) decompose language model activations into sparse features, but prompt transformations designed to preserve the underlying request can redistribute activity across them. A feature change observed on one transformed prompt need not recur, and even a recurring change need not affect a downstream computation. We introduce the redistribution sensitivity score (RSS), which identifies large feature responses that recur in a consistent direction, and evaluate the resulting supports on variants excluded from selection. Across 27 pretrained SAEs spanning eight language models, the tested prompt transformations produce a reproducible redistribution profile on these excluded variants. We then separate recurring representation change from signed readout contribution and decision failure. For a fixed linear readout, an exact decomposition shows how response magnitude, readout coefficients, directional alignment, and the clean margin determine downstream consequences. In two frozen safety readouts, combining RSS support erosion with the full clean margin yields recurrent failure AUROCs of 0.935 and 0.958 using median supports of 20 and 19 features, compared with 0.965 and 0.964 using full erosion. RSS therefore provides a way to identify recurring sparse representation changes and quantify their contribution to a downstream readout without equating representation change with failure. Code is available at https://github.com/OpenSAE/Redistribution-Sensitivity-Score

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.21240

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.21240 in a model README.md to link it from this page.

Datasets citing this paper 1

OpenSAE/SAEBase-Evaluation

Viewer •

Updated 6 days ago

•

12.1k

•

211

•

1

Spaces citing this paper 1

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co