Reducing private data leakage in fine-tuned LLMs via neuron pruning
In brief
- JEPA-Prune removes neurons storing private data without retraining.
- Pruning 1.6% of neurons cuts PII inference accuracy by up to 7.4x.
- Perplexity on unseen text stays close to the original model.
A new method, JEPA-Prune, helps limit private data leakage when fine-tuning large language models. It identifies and removes neurons responsible for memorizing personal information without degrading overall performance.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error