Factual Confidence Estimation in LLMs via Contrastive Likelihoods
In brief
- CFE/CFP method estimates factual confidence in LLMs using contrastive likelihoods.
- Significant improvement in calibration of confidence scores compared to existing approaches.
- Weak discrimination in low-resource languages limits practical utility.
This study introduces a method to estimate language models' confidence in factual claims by contrasting likelihoods between factual and non-factual conditions. Results show improved calibration of confidence scores, though discrimination remains weak in low-resource languages.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error