Learning to Read Out: Unembedding Dynamics in Language Model Pretraining
Learning to Read Out: Unembedding Dynamics in Language Model Pretraining
研究指出,語言模型的隱藏狀態可能已編碼任務資訊,但讀出層尚未能在輸出中呈現;因此,按輸出判定能力出現時間,反映的是讀出學習,而非模型獲得能力的時間。研究以 parameter-trajectory crosscoding 分析 Pythia 預訓練,發現進度達 0.4-1.1% 時,讀出開始重新組織,WordNet 詞類亦可從讀出行解碼。
Published on Oct 14
Authors:
,
,
,
,
Abstract
Hidden states encode task information before the readout expresses it, so behavioral timelines reflect readout learning rather than model acquisition, and faster readout training closes this gap earlier.
When does a language model acquire a capability? Developmental studies usually answer this by tracking when models start producing the correct token. But that prediction passes through the unembedding matrix, or readout, which is itself still being learned during pretraining. We show that a model's hidden states can encode task information before its own readout expresses it, so timelines read from outputs conflate what the model encodes with what its readout expresses. To track the readout directly, we introduce parameter-trajectory crosscoding, which fits one sparse dictionary to every token's readout row at every checkpoint. Unlike crosscoders on activations, it needs only checkpoint weights. We use it to locate a narrow early window, 0.4-1.1% of the way through Pythia's pretraining, in which the readout reorganizes and WordNet word categories become decodable from its rows. By then, linear probes on the hidden states already decode task information that the model's own outputs do not yet show. To test whether such gaps depend on how fast the readout learns, we pretrain small models from scratch that differ only in the readout's learning rate. We find that raising it makes the readout develop earlier and the gap close sooner, with little effect on final loss. Together, these results imply that early in training, where many developmental transitions are dated, a behavioral date can reflect when the readout learns to express task information and not when the model learns it.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.21025
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 2
Updated 9 minutes agohematteo/parameter-trajectory-crosscoders
Updated 9 minutes agohematteo/readout-recipe-control
Datasets citing this paper 1
hematteo/wu-crosscoder-snapshots
Spaces citing this paper 1
Collections including this paper 1
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co