2026 · AI / ML
wav2vec2 Layer Probing
A layer-by-layer study of spoken-digit and speaker information in wav2vec2.
- Speech recordings
- Layer-wise linear probes
- Pretraining comparison
Small-scale research reproductionGitHub ↗
Problem
Different layers of a speech model may encode different kinds of information.
My role
Designed and ran a layer-wise probing experiment with a random-weights comparison.
Approach
- Extract representations from 13 layers of wav2vec2-base.
- Train linear probes for spoken-digit and speaker classification.
- Repeat the analysis with randomly initialized weights to compare the effect of pretraining.
Results
- The repository provides a two-panel accuracy plot. The experiment uses one random seed and reports no error bars.
Stack
- Python
- PyTorch
- wav2vec2
- Jupyter