Why this is a common failure point
Multilingual agents usually break at the edges between languages rather than inside them. The agent answers a Spanish caller in English, switches languages mid-call when the transcriber mishears a word, or understands the question but replies with a translation that loses the meaning. Callers who mix languages, as many bilingual callers do, are the hardest case of all. Testing each language by hand is slow, so most teams test one or two and assume the rest work.How Roark tests it
Multilingual fans each attached flow out across the languages you pick. Every language runs the same flow with a native-speaking caller, so a failure in one language and not the others points straight at language handling. A code-switching option swaps in callers who mix their language with English mid-sentence.Setting it up
Pick the languages from the chip grid: Spanish, French, German, Portuguese, Italian, Dutch, Chinese, Japanese, Hindi, Arabic, Turkish, Greek, Indonesian, Thai, Tagalog, Malay and Hebrew. Each attached flow runs once per language, so 3 flows and 4 languages is 12 conversations per iteration. Turn on Code-switching to use callers who mix their language with English (for example, Spanglish).What it measures
The template seeds these, and you can add or remove metrics in Advanced before running.What to look for
- Compare languages, not totals. Filter the calls by language: a language that fails where the others pass is a language problem, not a flow problem.
- Wrong-language replies. Read the first agent turn of failing calls. An agent that opens in English and never switches fails everything after it.
- Code-switching is a separate test. Run it on its own so mixed-language failures don’t blur the single-language results.