LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs’ Probabilistic Beliefs

Researchers developed a method to analyze how large language models update their probabilistic beliefs, revealing that some models deviate from Bayesian updates and use heuristics that can outperform exact Bayesian methods in task performance. This suggests LLMs' internal probabilistic models may be misspecified, and the method can help diagnose issues in LLM-based inference systems.