Language Dominance in AI: EPFL Study Reveals English BiasEPFL Researchers Uncover Biases in Large Language Models

Researchers at EPFL have discovered that large language models, such as Open AI's ChatGPT and Google's Gemini, primarily rely on English when processing information, even when prompted in a different language.

This could have significant implications for linguistic and cultural biases as AI becomes more integrated into our lives.

Despite the ability of these models to interact in various languages, they are trained on predominantly English text, with billions of text parameters. Some have theorized that they translate to the target language at the last moment, but there has been little evidence to support this until now.

The EPFL researchers, from the Data Science Laboratory (DLAB), studied the Large Language Model Meta AI (Llama-2) to determine which languages were being used at different stages of the computational process. They found that the model often predicts English words even when prompted in a different language.

The models work by predicting the next word based on a vector of numbers that represent the word. Professor Robert West, head of DLAB, explains that the models use 80 layers of computational blocks to transform one vector into another, with the number of calculations determining the model's power and accuracy.

In their paper, "Do Llamas Work in English? On the Latent Language of Multilingual Transformers," the researchers forced the model to answer after each layer, revealing that it often predicted English words even when prompted in a different language. This led the researchers to theorize that the model relies on a representation of concepts that are biased towards English.

West suggests that this English dominance may have consequences for how we think about the world, as language shapes our perception of reality. He recommends further research into the psychological effects of language models and the potential biases they may have.

While it may be tempting to simply feed English content into the model and translate it back to the desired language, West warns that this may result in a loss of nuance and potentially reinforce linguistic and cultural biases. Therefore, it is important to continue studying and addressing these issues as AI becomes more prevalent in our lives.

Alton Shaffer
With a strong hold on the business domain with an experience of over 10 years, Alton owns a Masters Degree in Business Administrations. He is associated with Industry News USA from last 3 years and handles the Business section successfully. In team Industry News USA, Alton is known as an affluent source of knowledge. He is honored for greatest efforts in his field. With a unique style of communicating, Alton is labeled as a problem-solver of Industry News USA.