Sparse Autoencoders to Mitigate Catastrophic Forgetting in Continual Learning
In brief
- SAGA leverages sparse autoencoders to enforce local geometric constraints in continual learning.
- The approach reduces representation drift while maintaining model plasticity.
- Accuracy gains of 3 to 12 percentage points on three benchmarks with pre-trained ViT-B/16 models.
Researchers introduce SAGA, a method using sparse autoencoders to constrain representation drift in supervised continual learning. The locally induced geometric constraints improve stability without sacrificing plasticity. Results show significant gains across three benchmarks with pre-trained ViT-B/16 models.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error