Consistent Concept Discovery with Sparse Coding Beyond SAEs
In brief
- ConSDL replaces SAEs' linear encoder with per-sample sparse coding.
- The method improves dictionary and coefficient consistency by 17.6% to 35.9%.
- Superior performance on 16 synthetic setups and 4 vision/language benchmarks.
A new method, ConSDL, enhances the reliability of analyses based on sparse autoencoders (SAEs) by using per-sample sparse coding instead of a standard linear encoder. Results show improved dictionary and coefficient consistency, with significant gains on visual and language benchmarks.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error