Artifacts you can cite

Papers

• MultiIdiom: A Multilingual Idiom Dataset with Span Annotations across Seven Typologically Diverse Languages

  Dataset paperIn preparation

  Introduces MultiIdiom itself: how the 112,317-example, 14,505-idiom, seven-language corpus was built — a six-stage LLM-assisted pipeline that generates sense inventories from Wiktionary seeds, labels idiomaticity and sociolinguistic metadata, and expands into morphologically varied example sentences with character-level spans. Argues the dataset is the first public Telugu idiom-detection resource with in-context idiomaticity and span labels, one of few Spanish resources pairing sense definitions with in-context examples, and the first span-annotated benchmark spanning Indo-European, Dravidian, and Austronesian language families in one dataset.

  Not yet submitted, so there is nothing to cite here. The dataset below is published and carries its own DOI — cite that in the meantime.

• IdiomBERT: A Joint Multilingual Benchmark for Idiom Span Extraction and Classification, and When Fine-Tuned Models Rival LLMs Zero-Shot

  Modeling paperIn preparation

  Evaluates the harder, joint version of idiom detection — locating the span and judging idiomaticity together, in one pass, rather than the two treated separately in prior work. Compares eleven systems (mBERT pipeline and joint architectures, a BIO diagnostic, and zero/few-shot GPT-4o, Llama-3.3-70B, and Kimi-k2.5) across five languages plus two held-out zero-shot languages, using MultiIdiom as the benchmark and a single pooled statistical correction across every tested language cell. On silver-standard labels, fine-tuned mBERT significantly outperforms GPT-4o zero-shot on three held-out languages and Llama-3.3-70B on all seven — but on the two languages checked against real human annotation, that advantage does not hold up, so the paper frames its finding as parity with far larger zero-shot models at a fraction of the inference cost, not a proven win on human-verified data. A second part of the paper shows the assumed advantage of QA-style span extraction over BIO tagging is mostly a tokenizer artifact, and proposes scoring joint errors per example rather than by aggregating subtask scores separately, which a prior metric does in a way that overstates joint capability by 20 to 21 points.

  Not yet submitted, so there is nothing to cite here. The dataset below is published and carries its own DOI — cite that in the meantime.

Dataset

Maddineni, S. (2026). MultiIdiom: A Multilingual Idiom Detection Dataset with Joint Idiomaticity and Span Annotation [Data set]. Zenodo. https://doi.org/10.5281/zenodo.20774989

@dataset{maddineni_multiidiom_2026,
  author       = {Maddineni, Shishir},
  title        = {{MultiIdiom: A Multilingual Idiom Detection Dataset
                   with Joint Idiomaticity and Span Annotation}},
  year         = {2026},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.20774989},
  url          = {https://doi.org/10.5281/zenodo.20774989},
  license      = {CC-BY-SA-4.0}
}

Models

• IdiomBERT (System E) — joint

  Joint idiomaticity classification + span start/end localization · trained on MultiIdiom — EN, ES, HI, TE · Justarandomperson/IdiomBERT-system-e

  One forward pass yields a literal/idiomatic label and the span indices. Task heads ship separately in task_heads.pt — see the model card for the exact loading procedure. Powers the extractor’s "Literal vs. idiomatic" method — the only one that judges sentences with no idiom at all, not just the ones with one.

• Earlier prototype — BIO token classifier

  BIO sequence labelling (O / B-IDIOM / I-IDIOM) · trained on ID10M (Tedeschi et al., 2022)

  A LoRA adapter over mBERT, fine-tuned on the public ID10M dataset — predates MultiIdiom and is not the model the research below evaluates. It is the default "Model" method in the extractor today because it is the only full BIO checkpoint that still exists as deployable weights. Structurally has no literal/idiomatic decision: it flags a span or it doesn’t.

• System G — BIO token classifier

  BIO sequence labelling (O / B-IDIOM / I-IDIOM) · trained on MultiIdiom — EN, ES, HI, TE

  Full fine-tune (not LoRA), trained and evaluated on MultiIdiom — this is the checkpoint the accuracy figures on this site and in the paper refer to. Not currently deployed: the trained weights exist only as saved predictions, not as a loadable checkpoint, so it cannot serve the live extractor until retrained or recovered.

Earlier corpora

• Idiomator Multilingual Idiom Example Dataset

  Earlier collection: idiomatic sentences scraped from OPUS subtitles plus Wiktionary idiom lists. Examples are not aligned to specific idioms — superseded by MultiIdiom.