LPHI-mRAG
Language preference and hallucination in multilingual retrieval-augmented generation
Do language models prefer some languages over others when answering from documents? We test this for chatbots that answer questions from a collection of documents in several languages.
Chatbots that answer questions from your own documents (retrieval-augmented generation) first search for relevant texts and then let a language model write an answer from them. What the search returns strongly shapes the answer, and language models sometimes make things up (hallucinate).
The effect of how many documents are given, and in which order, has been studied. The effect of their language has not. In a multilingual country like Switzerland, the useful document is often in a different language from the question. We study whether language models, although presented as multilingual, prefer some languages, and whether this leads to more hallucinations.
Publications
- Are Language Models Script-Aware?
- Script Choice in LLMs: Evidence for Late-Layer Commitment
- Are the LLMs Capable of Maintaining at Least the Language Genus?