MEXA | Nafiseh Nikeghbal

MEXA assesses the multilingual capabilities of pre-trained, English-centric LLMs using parallel sentences — available for many more languages than existing downstream tasks. Leveraging the observation that English-centric models use English as a pivot in their intermediate layers, MEXA computes the alignment between English and non-English languages to estimate multilingual performance (Kargaran et al., 2025).

In its default setting, MEXA reaches a statistically significant average Pearson correlation of 0.90 with three established downstream tasks across nine models and two parallel datasets.

📄 Paper: ACL Anthology · arXiv:2410.05873
💻 Code: github.com/cisnlp/MEXA
🏆 Leaderboard: huggingface.co/spaces/cis-lmu/Mexa

References

2025

ACL

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment

Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, and 3 more authors

In Findings of the Association for Computational Linguistics: ACL 2025, Jul 2025

Abs arXiv Bib HTML Code

@inproceedings{kargaran2025mexa,
  title = {MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment},
  author = {Kargaran, Amir Hossein and Modarressi, Ali and Nikeghbal, Nafiseh and Diesner, Jana and Yvon, Fran{\c{c}}ois and Sch{\"u}tze, Hinrich},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  pages = {27001--27023},
  year = {2025},
  month = jul,
  publisher = {Association for Computational Linguistics},
  address = {Vienna, Austria},
}