TempUN

XME: Cross-lingual Editing in Multilingual Language Models

Indian Institute of Technology Gandhinagar, India 🇮🇳
EACL 2024 (Malta 🇲🇹)
^*Indicates Equal Contribution

Abstract

The training of large language models (LLMs) necessitates substantial data and computational resources, and updating outdated LLMs entails significant efforts and resources. While numerous model editing techniques (METs) have emerged to efficiently update model outputs without retraining, their effectiveness in multilingual LLMs, where knowledge is stored in diverse languages, remains an underexplored research area. This research paper introduces the cross-lingual model editing (XME) paradigm, wherein a fact is edited in one language, and the subsequent update propagation is observed across other languages. To investigate the XME paradigm, we conducted experiments using BLOOM, mBERT, and XLM-RoBERTa using the two writing scripts: Latin (English, French, and Spanish) and Indic (Hindi, Gujarati, and Bengali). The results reveal notable performance limitations of state-of-the-art METs under the XME setting, mainly when the languages involved belong to two distinct script families. These findings highlight the need for further research and development of XME techniques to address these challenges. For more comprehensive information, the dataset used in this research and the associated code are publicly available at the following [URL].

Contributions and Findings 💡

Figure 2: An outline for hypernetwork-based model editing technique.

Table 1: Dataset statistics in different languages. Note: TFR and VFR are the average length of training-filtered and validation-filtered rephrases, respectively. Inv_bloom and Inv_xlm are the inverse proportion of BLOOM and XLM-RoBERTa. Lastly, in all the languages, the size of validation and test remains 10444 and 1193, respectively.

Table 2: The table represents G_S for fine-tuned mBERT (left) and BLOOM (right) on ‘en’ dataset using MEND.

Table 3: The table represents S_S for fine-tuned mBERT on the ‘en’ (left) and ‘hi’ (right) dataset using MEND.

Figure 3: The figure illustrates G_S given the editing language (x-axis) and fine-tuning languages (y-axis) for all the three models BLOOM (left), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.

Figure 4: The figure illustrates G_S (Left) and G_S (Right) given the editing language (x-axis) and fine-tuning datasets (y-axis) for all the three models BLOOM (top), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.

BibTeX

@inproceedings{beniwal-etal-2024-cross, title = "Cross-lingual Editing in Multilingual Language Models", author = "Beniwal, Himanshu and D, Kowsik and Singh, Mayank", editor = "Graham, Yvette and Purver, Matthew", booktitle = "Findings of the Association for Computational Linguistics: EACL 2024", month = mar, year = "2024", address = "St. Julian{'}s, Malta", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2024.findings-eacl.140", pages = "2078--2128", abstract = "The training of large language models (LLMs) necessitates substantial data and computational resources, and updating outdated LLMs entails significant efforts and resources. While numerous model editing techniques (METs) have emerged to efficiently update model outputs without retraining, their effectiveness in multilingual LLMs, where knowledge is stored in diverse languages, remains an underexplored research area. This research paper introduces the cross-lingual model editing (XME) paradigm, wherein a fact is edited in one language, and the subsequent update propagation is observed across other languages. To investigate the XME paradigm, we conducted experiments using BLOOM, mBERT, and XLM-RoBERTa using the two writing scripts: Latin (English, French, and Spanish) and Indic (Hindi, Gujarati, and Bengali). The results reveal notable performance limitations of state-of-the-art METs under the XME setting, mainly when the languages involved belong to two distinct script families. These findings highlight the need for further research and development of XME techniques to address these challenges. For more comprehensive information, the dataset used in this research and the associated code are publicly available at the following [URL](https://github.com/lingo-iitgn/XME).", }

XME: Cross-lingual Editing in Multilingual Language Models

A short overview of the our work on the Cross-lingual Model Editing (XME).

Abstract

What is "Cross-lingual Model Editing"?

Figure 1: XME pipeline: we update a fact in one language (say English) and check whether the same fact is updated in different languages.

Contributions and Findings 💡

Figure 2: An outline for hypernetwork-based model editing technique.

Table 2: The table represents G_S for fine-tuned mBERT (left) and BLOOM (right) on ‘en’ dataset using MEND.

Table 3: The table represents S_S for fine-tuned mBERT on the ‘en’ (left) and ‘hi’ (right) dataset using MEND.

Figure 3: The figure illustrates G_S given the editing language (x-axis) and fine-tuning languages (y-axis) for all the three models BLOOM (left), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.

Figure 4: The figure illustrates G_S (Left) and G_S (Right) given the editing language (x-axis) and fine-tuning datasets (y-axis) for all the three models BLOOM (top), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.

Detailed Video Presentation

Poster

BibTeX

XME: Cross-lingual Editing in Multilingual Language Models

A short overview of the our work on the Cross-lingual Model Editing (XME).

Abstract

What is "Cross-lingual Model Editing"?

Figure 1: XME pipeline: we update a fact in one language (say English) and check whether the same fact is updated in different languages.

Contributions and Findings 💡

Figure 2: An outline for hypernetwork-based model editing technique.

Table 2: The table represents GS for fine-tuned mBERT (left) and BLOOM (right) on ‘en’ dataset using MEND.

Table 3: The table represents SS for fine-tuned mBERT on the ‘en’ (left) and ‘hi’ (right) dataset using MEND.

Figure 3: The figure illustrates GS given the editing language (x-axis) and fine-tuning languages (y-axis) for all the three models BLOOM (left), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.

Figure 4: The figure illustrates GS (Left) and GS (Right) given the editing language (x-axis) and fine-tuning datasets (y-axis) for all the three models BLOOM (top), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.

Detailed Video Presentation

Poster

BibTeX

Table 2: The table represents G_S for fine-tuned mBERT (left) and BLOOM (right) on ‘en’ dataset using MEND.

Table 3: The table represents S_S for fine-tuned mBERT on the ‘en’ (left) and ‘hi’ (right) dataset using MEND.

Figure 3: The figure illustrates G_S given the editing language (x-axis) and fine-tuning languages (y-axis) for all the three models BLOOM (left), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.

Figure 4: The figure illustrates G_S (Left) and G_S (Right) given the editing language (x-axis) and fine-tuning datasets (y-axis) for all the three models BLOOM (top), mBERT (middle) and XLM-RoBERTa (right) when edited using MEND.