Intochat Test Finds Leading Chatbots Understand Swiss German but Still Invent Promises to Customers

Large language models from Anthropic, Google and OpenAI now cope well with Swiss German, but they still make up details when answering customers, according to a test by Intochat reported by Inside IT on October 30.

Intochat tested five models, Claude Sonnet 5.5 and Claude Haiku 5.5 from Anthropic, Gemini 3.8 Flash and Gemini 3.1 Pro from Google, and OpenAI’s GPT-6 Luna. It sent them 50 typical customer messages written in Swiss German and in standard German, then compared the answers.

Understanding was no longer the problem. The models captured between 97.8% and 100% of the core information in dialect messages, and between 99.3% and 100% in standard German. Claude Sonnet 5.5 and both Gemini models scored a full 100% in Swiss German, GPT-6 Luna 99.3% and Claude Haiku 5.5 97.8%. GPT-6 Luna also replied in dialect in 94% of cases.

What it means in practice

The bigger risk for businesses was invented commitments, and that barely varied by language. GPT-6 Luna did clearly best on standard German queries, with 4% of its answers containing made-up details. About 10% of Claude Sonnet 5.5’s answers were wrong and padded with invented information, and the figure was 14% for both Gemini 3.8 Flash and Claude Haiku 5.5. Gemini 3.1 Pro came last, with 20% wrong answers and a further 16% partly invented. The fabrications were mostly small promises the business had never made, such as assuring a hotel guest there would be room for skis, or telling a customer that a different size could simply be requested with a return. Customers can easily read such lines as a binding commitment.

The results matter for Swiss firms putting chatbots in front of customers, where trust is already limited: a Visa survey found nearly 70 percent of Swiss have used AI for shopping but few trust it to pay. Software teams are also moving fast, with a survey showing nearly nine in ten firms now code with AI. Public bodies have taken a more controlled route, as when Ticino moved administrative translation to the Swiss model Apertus.

What to watch next is whether companies add checks that stop chatbots from making commitments on their own, whether Intochat publishes its full method and prompts, and whether similar results hold for French- and Italian-speaking customers.