1st Luxembourgish Language Machine Learning Model LuxemBERT Created

The first Luxembourgish language model using the BERT (Bidirectional Encoder Representations from Transformers) technology, LuxemBERT, was recently created.

BERT is a machine learning technique for natural language processing pre-training developed by Google in 2018.

The LuxemBERT language model, which is open source and publicly available online, was developed through a partnership between BGL BNP Paribas and the University of Luxembourg’s Interdisciplinary Centre for Security, Reliability and Trust (SnT).

Language models powers the artificial intelligence (AI) driven technology like translation software, chatbots and text recognition. As it takes significant investment and time to create a functional language model, the common practice when building AI systems like chatbots is to use a pretrained model publicly available from the research community. The challenge is that niche languages do not have enough available resources, so there was no model available for Luxembourgish, until now.

As a result, the collaboration between BGL BNP Paribas and SnT, and with the support of the Alphonse Weicker Foundation, an innovative approach to tackle the challenge of training models in low-resource languages was created. The project was led by Prof Jacques Klein at SnT. LuxemBERT will help language-based AI systems and will be a key step in developing multilingual virtual assistants, which enhance customer experience. Using the power of AI, the virtual assistant can understand the customer’s question and provide the most relevant answer.

“Creating solutions based on the most innovative technologies is key for our business and we are proud to have contributed to the development of a Luxembourgish language model. This will no doubt have a positive impact on the country’s AI innovation ecosystem”, said Béatrice Belorgey, Chair of the BGL BNP Paribas Executive Committee and Country Head of the BNP Paribas Group in Luxembourg.

“The result of FinTech innovations for individual customers is that most of their interactions with their banks now happen online, making a chatbot a vital resource for them to conduct their transactions. BGL BNP Paribas had the foresight to see the need for a Luxembourgish language model and brought this challenge to us because of our considerable experience with natural language processing,” said Prof Björn Ottersten, Director of SnT. “Now that LuxemBERT has been released publicly, other actors in the AI domain can begin building systems with it and we look forward to being the academic stewards of this model as it grows in its usage.”