ToPA: Block-wise Toeplitz Adaptation for Efficient Fine-Tuning
In brief
- ToPA leverages block-wise Toeplitz matrices for parameter-efficient fine-tuning.
- The method improves expressivity and stabilizes gradients compared to existing approaches.
- Validated on 21 NLP and CV datasets across 5 model architectures.
This study introduces ToPA, a parameter-efficient fine-tuning method based on block-wise Toeplitz matrices. It enhances expressivity and gradient stability compared to existing approaches while reducing rank or sparsity constraints. Results on 21 NLP and CV datasets confirm its effectiveness.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error