About

About Idiomator

Idiomator is a multilingual idiom-detection tool. It finds idiomatic expressions in text — pasted, uploaded as a PDF, or looked up directly — across 7 languages: English, Spanish, Hindi, Telugu, German, French, Indonesian.

* Idiom detection is hard, even for humans. Accuracy varies by language and by how unusual the phrasing is — see the numbers below rather than take that on faith.

Why We Created Idiomator

One of the biggest barriers in language learning is not knowing what you don’t know. Idioms often fall into that blind spot — you can know every word in a sentence and still miss the meaning. Idiomator exists to surface that hidden knowledge, tailored to your actual text.

What’s Next

The dataset is published and the extractor is live. Both papers — one on the dataset, one on the models — are now under ARR review as anonymous submissions; drafts are linked on publications, with the full methods detailed on the research page. On the modeling side we’re looking at closing the gap between our silver-label results and the human-verified ones (see the IdiomBERT paper summary), and at extending the zero-shot languages toward full training coverage.

Who Built This

I’m Shishir Maddineni, a student developer and language enthusiast. I’ve always struggled with idioms while learning languages — there’s no easy way to master them. So I built Idiomator to make that process faster, and built the dataset and models behind it because the resources to do it properly didn’t already exist for most of these languages.

Our Dataset

Idiomator runs on MultiIdiom, a dataset we built in-house: 14,505 idioms across 34,184 senses and 112,317 examples, in 7 languages. It’s published and citable now, under CC BY-SA 4.0 — 10.5281/zenodo.20774989. Full construction method and the paper describing it are on the research page. If you’d like to help validate idioms in your native language, get in touch.

Our Models

Three mBERT models exist, at different stages. Not all of them run the same checkpoint their names might suggest, so we’re specific about which is which:

IdiomBERT (System E) — joint — live, beta method — Joint idiomaticity classification + span start/end localization. Trained on MultiIdiom — EN, ES, HI, TE. One forward pass yields a literal/idiomatic label and the span indices. Task heads ship separately in task_heads.pt — see the model card for the exact loading procedure. Powers the extractor’s "Literal vs. idiomatic" method — the only one that judges sentences with no idiom at all, not just the ones with one.

Earlier prototype — BIO token classifier — live, default method — BIO sequence labelling (O / B-IDIOM / I-IDIOM). Trained on ID10M (Tedeschi et al., 2022). A LoRA adapter over mBERT, fine-tuned on the public ID10M dataset — predates MultiIdiom and is not the model the research below evaluates. It is the default "Model" method in the extractor today because it is the only full BIO checkpoint that still exists as deployable weights. Structurally has no literal/idiomatic decision: it flags a span or it doesn’t.

System G — BIO token classifier — not deployed — BIO sequence labelling (O / B-IDIOM / I-IDIOM). Trained on MultiIdiom — EN, ES, HI, TE. Full fine-tune (not LoRA), trained and evaluated on MultiIdiom — this is the checkpoint the accuracy figures on this site and in the paper refer to. Not currently deployed: the trained weights exist only as saved predictions, not as a loadable checkpoint, so it cannot serve the live extractor until retrained or recovered.

The model behind the default extractor method today is the earlier ID10M-trained prototype, not the MultiIdiom-trained System G that the research below evaluates — System G exists and has real numbers, but isn’t currently loadable as deployable weights. Full accuracy figures for the models that are live, and System G’s numbers in their own context, are on the research page.

Acknowledgements

Idiomator’s earliest prototype was fine-tuned on the ID10M dataset and drew on Wiktionary’s rich gloss data. The current system is trained on our own MultiIdiom dataset, but we remain grateful to the broader MWE and language-learning communities whose open work made this project possible.