MultiIdiomDataset
14,505 idioms, 34,184 senses and 112,317 examples across seven languages, each occurrence labelled idiomatic or literal and marked with a character-level span.
Idioms are the cultural heart of any language.
They carry a culture’s humour, history and way of seeing the world, and they are exactly what dictionaries, translators and AI models get wrong, most of all in the languages technology leaves out.
Idiomator builds open data and free tools so the idioms of every language can be understood, by people and by machines.
- spill the beans (English: literally “spill the beans”, meaning reveal a secret)
- अपना उल्लू सीधा करना (Hindi: literally “straighten your own owl”, meaning act in your own self-interest)
- tomar el pelo (Spanish: literally “take the hair”, meaning tease someone, pull their leg)
- ada udang di balik batu (Indonesian: literally “there’s a shrimp behind the rock”, meaning there’s a hidden motive)
- ins Gras beißen (German: literally “bite into the grass”, meaning die)
- ఆటలో అరటిపండు (Telugu: literally “a banana in the game”, meaning someone treated as insignificant)
- coûter les yeux de la tête (French: literally “cost the eyes of the head”, meaning be extremely expensive)
Cultural exposure, for people and for machines
Most language technology is built on a handful of well-resourced languages. Everyone else’s idioms get translated word for word, or not at all. Equal access to a language’s figurative heart is a question of cultural equality.
For people
A learner, a translator, someone reconnecting with a grandparent’s language: each should be able to find what an idiom really means, with real examples, whatever language it comes from.
For machines
Models learn culture from data. When a language’s idioms are missing from that data, the model flattens its culture into literal words. Open, labelled idiom data is how that changes.
Seven languages mapped. More than 7,000 to go.
Four languages are trained on; three are held out entirely to test whether idiom knowledge transfers to languages a model has never seen labelled. We mark which is which rather than implying they perform alike.
Even our own data is unequal
English has 9,643 idioms in MultiIdiom; Indonesian has 46, a 209× gap. The imbalance on the open web is the imbalance in the data, unless someone corrects it on purpose. That is the work.
Full split sizes and agreement figures are on the research page.
Some phrases don’t mean
what they say.
Kick the bucket has nothing to do with buckets. You can’t work an idiom out from its parts, so a dictionary that indexes words misses it and a translator renders it literally: confidently, and wrong.
Idiomator finds idioms in seven languages and tells you what they actually mean. It runs on an open, span-annotated dataset built for exactly this problem.
The same five words
He finally spilled the beans about the surprise party.
Idiomaticrevealed a secret
Nothing here is about beans.
Three things learners try. All three break.
Idioms are non-compositional: the meaning is not a function of the parts. Every ordinary strategy for decoding unfamiliar language assumes it is.
Look it up
A dictionary indexes words, not phrases.
Search “bucket” and you get a container. The idiom is a property of the whole phrase, so word-level lookup can’t reach it, and learners rarely know they should be searching for a phrase at all.
Translate it
Machine translation renders it literally, and confidently.
There is no uncertainty flag on a wrong idiom. The output is fluent, grammatical, and about the wrong subject, which is harder to catch than an obvious error.
Infer it from context
The same phrase is idiomatic in one sentence and literal in the next.
Recognising that you are even looking at an idiom is itself a task. That is why the dataset labels idiomaticity per occurrence, not per phrase.
Small things we’re doing now
Publishing the data openly
MultiIdiom is CC BY-SA 4.0 with a DOI (10.5281/zenodo.20774989). Anyone can build on it without asking.
Checking labels with native speakers
Native speakers have been reviewing Telugu, Hindi and French annotations. A model shouldn’t be the only judge of a culture’s idioms.
Testing languages the model has never seen
German, French and Indonesian are held out of training entirely, to learn whether idiom knowledge transfers, so the next language doesn’t start from zero.
Keeping the tools free
The extractor and dictionary need no account and cost nothing.
Taking idioms from the people who use them
Anyone can submit an idiom from their own language.
What we built instead
A corpus, a model trained on it, and two tools you can use right now. All of it open, all of it citable.
ExtractorTool
Paste text or upload a PDF. Idioms come back marked in place, with a rule-based pass you can run alongside the model to compare.
DictionaryTool
Every sense, with definitions, register and region, plus usage examples drawn from parallel corpora with the idiom highlighted.