A new study by researchers from Technion, Google Research, and Apple reveals that large language models (LLMs) have a deeper understanding of truthfulness than previously thought. The researchers analyzed the internal representations of LLMs across various datasets and found that truthfulness information is concentrated in specific response tokens, VentureBeat reports. By training classifier models on these tokens, they were able to predict errors and error types, suggesting LLMs encode information related to their own truthfulness. The study also uncovered a discrepancy between the models’ internal activations and their external outputs, indicating that current evaluation methods may not accurately reflect LLMs’ true capabilities.