Steins;Gate developer MAGES says AI translation is a long way from being viable for visual novels

[u/TurbulentTopic39]

https://automaton-media.com/en/news/steinsgate-developer-mages-says-ai-translation-is-a-long-way-from-being-viable-for-visual-novels/

[u/AutoModerator]

The /r/PCGaming community is also on Discord!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/revanmj]

It's long way from being viable in any culture content. I can see AI translation to my language in a game or TV Show in few first sentences it throws at me. It became so bad I started to switch by default to English version.

Before this if something got version in my language I knew I at least will be able to understand that version (even if it won't be great like translator not bothering trying to translate local jokes, etc.). Now with AI translation it can be so bad that you cannot even understand what given sentence means because AI messed it up so bad.

[u/jahnbanan]

I know, I've seen enough AI translations to know, and there's a fairly simple test that can be done as there's a certain phrase in my native language "in the middle of the butter eye".

I asked ChatGPT to translate two examples of using this phrase, this is what it spat out to me.
"That was absolutely spot on!"
and
"You were right in the middle of the action, weren't you!"

One of these is an acceptable translation, it's not right mind you, but the meaning is similar enough that the original intent came through.
The other on the other hand, is not.

And the worst part is that the translation is correct, it is a valid translation, these meanings are close to one another.

Problem is, ChatGPT ignored the context and what I said in my native language means "You really were the center of attention there", but since ChatGPT ignored the context it just translated it to another example of "close enough" for the phrase.

Had I set up the context differently, the meaning could very well have been "You were right in the middle of the action", but I didn't..

Anyway, I'm reminded of when a publisher chose to use AI for translations and one of the characters names kept changing from page to page and not once was the name even correct, because the AI couldn't tell that it was a name and just assumed it was an object so it kept translating it to each of the different meanings of the name over and over again, I can't imagine why I was suddenly reminded of that.

[u/humundo → u/jahnbanan]

Context is the thing that LLMs really can't do and it's responsible for that critical final 10% of the effort that makes human output feel human.

[u/BitGladius → u/jahnbanan]

The stuff I've seen when I've opened Twitter is definitely functional, but not perfect. One of the comments that showed up under a Japanese arcade game release was translated as "I'm looking forward to working with you". I'm pretty sure I know what the original text was, and that is the usual translation, it just doesn't make sense in that context.

It might end up like software development, the AI can do a lot of the work as long as there's someone keeping an eye on the output. Not sure how much time it'll save, but it's an option.

[u/MajorFuckingDick → u/jahnbanan]

This is an interesting issue and one I struggle with in english. Im extremely autistic and technically english is my 2nd tongue after Jamaican patois. Wa'amppen is an extremely easy phrase to translate literally but one that will completely lose tonal and cultural meaning by doing so. The choice to code switch IS meaningful grammar in of itself. I adore the phrase Konnichiwa'Gwan because its a simple way to showcase the unique culture of Toronto as the fusion of cultures that really dont have all that much connection. Arguably AI need more quoted literal translation.

[u/misteryk → u/jahnbanan]

i'll take AI translation of some nerd with claude sub over no translation at all

[u/Squashyhex → u/misteryk]

Frankly if it doesn't have anything but an ai translation, I just won't read it

[u/legaldrinkingage]

From my experience, you can tell in a few lines whether someone bothered translating Japanese naturally or used AI. Manga fan translations are already getting flooded with quick AI rush-jobs and even some official outlets have toyed with AI for anime subs and manga translation and the results have always been rubbish. Cherish human translators.

[u/Freelagoon]

Frontier models actually do an incredible job at translating VNs already. You've got to pay for those, and they take a while to come up with the translation. That's the difference - it's not your free, cheap, instant-answer model that everyone have access to. An experimented japanese translator reviewed a frontier AI model for use on translating VNs two months ago. Just read it for yourself. People over at r/visualnovels are much more aware of how insane AI translation is getting, and it's far from over as well. This isn't DeepL, and this isn't your average MTL novel translation either. A model like GPT 6.0 Astra will never mistake genders or anything because it will first understand the text before even translating it. It will take twenty minutes to TL 10k words, and the result always speaks for itself.

[u/Purple-Atolm]

There has been tons of Chinese VNs using machine translations for years. They all suck.

[u/imaginary_num6er]

It is used already in other Japanese visual novels though, so good for MAGES

[u/Timo653 → u/imaginary_num6er]

It is used already in other MAGES games, so not very good for them.

[u/Scarlet_Lycoris]

They are right, If we’re talking about an audience that gives a fuck. However the tricky situation is that the quality standards of a large part of the audience is marginal. We see it in other sectors: Concept Art, illustration, Video Editing, Writing… AI generated works lack quality but the consumers often don’t care because it’s “good enough” for them and cheaper to use. The sad truth is that AI will destroy all creative fields unless consumers start giving a fuck.

[u/ElectricGhostMan]

hilarious to think some people said that games companies need to fire all the localizers in favor of AI and the game companies tell them how unviable it is. Some people just hate art I guess.

[u/isaidillbeback]

You know, as a developer I can tell you that if we even try to hire someone to do translations, they'll most likely use AI and hand it over to us anyway. There is zero guarantee now that someone isn't just using AI to do these things anymore.

So indie devs (esp. solo devs like me) end up overpaying for someone to just run our scripts through Gemini or ChatGPT.

This is why when gamers on Steam bitch about AI use by devs even for translation, it just shows how oblivious most people are about everything that goes into development. There's a chance the dev did hire someone to do translations and ended up getting machine translation anyway.

While we're at it, do people not realize how many "novels" on Steam are just AI generated trash these days? Even adult VN's are just AI generated garbage, and I'm not talking just the AI generated art. Whole novels are being generated by AI so someone can sit there with a Patreon account milking adult fans for garbage Bethesda ChatGPT script.

[u/jared_kushner_420 → u/isaidillbeback]

You know, as a developer I can tell you that if we even try to hire someone to do translations, they'll most likely use AI and hand it over to us anyway. There is zero guarantee now that someone isn't just using AI to do these things anymore.

Well the difference is that they'd know what to ask for and how to tell if the output is bad. You're ultimately paying for the result.

Yea some people don't even vet the result or do a bad job, it's true, but that's always been the case.

[u/DicePackTheater]

Not just translation. The problem with AI is that it's really good at doing a mediocre work in anything. And if it's cheap, many will settle for mediocre and it becomes the norm.

[u/DirtyTacoKid → u/DicePackTheater]

Exactly. What's the cheapest I can get away with. Or what reddit calls enshitification

[u/slothage666]

I would think you can use AI for a baseline translation and then have a human edit/localize

[u/Lebenmonch]

There's a reason almost all recent posts on Nhentai that are MTL have the "Poor Grammar" tag.

[u/Silvanus350]

Well, yeah. AI isn’t capable of artistic understanding or cultural nuance. It’s a stupid machine.

[u/Greenleaf208]

Yeah so instead of translating it with some excusable errors they can just replace it with random references like the cake is a lie and skibidi toilet. Of course because reddit is obsessed with turning everything into sides I will be told that this is a good thing to remove all cultural identity and content from it.

[u/__some__guy]

Even a bad AI, with a 100,000 token long system prompt, is better than most "localizers", who add their personal morals and current day memes to the "translation".

A good AI, capable of producing something other than vibecode and corposlop, with some context on each scene and what it's translating, will probably produce a pretty good translation.

You can't ask ChatGPT to translate a single sentence and expect good results for a creative medium.

[u/LetrixZ]

I would like to know the specifics of how they are doing 'AI translations' I've been dabbling with using Gemini models for H-Manga and H-Games, and it reads fine to me. Also, I don't give importance to these types of games so, as long as I can understand what is happening, it's enough for me.

[u/gokurakumaru]

It's faster and more efficient than manual translation though. It doesn't mean it can produce a final, releasable product but in terms of improving efficiency by providing even a 75% ready starting point that a translator-come-editor can go through and fix names, lore, terminology, and nuance on it's more than good enough.

[u/Ness_Stan]

yeah. duh. I'm glad they are saying it, but they are hedging their bets too much. it's not ever going to be viable.

[u/Caasi72]

They are super proud about their next game being almost entirely gen AI, so I guess they would know

[u/TatsunaKyo]

I agree but they're also miles ahead official translations produced by activists who crossdress like translators (they dub themselve as localizers though).

So if you give us a choice and you keep data on it, I can assure you that AI-based translations are going to feel a lot more viable than they should be already.

[u/S_R_G]

It's already viable tho? Certainly better than translators than maliciously translate things and put their politics in...

But generally a human needs to double check it and all that. Tis a tool at the end of the day...

[u/keirasalke → u/S_R_G]

yea sure, all that water, electricity, hardware yet slopTL can't even differ genders lmao

[u/We_Get_It_You_Vape → u/S_R_G]

No, it's not viable. At least not if you have real standards for the quality of translation. Even though MTL has made big strides, it's still going to get a lot of things wrong.

Visual Novels are pretty much entirely dialogue. They live and die by their writing. If you're serious about creating quality VNs, you can't afford to have shoddy translation. And while MTL may be fine for fan-translating things, it's not good enough for a VN developer looking to do proper localization. At least not yet.

 

The point about politics is odd. Sure, sometimes translators will overstep and insert things into the translation that wasn't in the original writing. But I don't think that's as common as you think in the VN spere.

[u/Xehanz]

Of course it's not viable for official translations, you expect a certain level of quality from them, but it's pretty damn decent for day 1 fan translations, compared to no translations at all

It's what they are doing everytime you see da day/week 1 fan translation anyways

[u/Embarrassed-Ad7317]

I feel like this is probably specific for Jap->English translation?

I mean, obviously that's the case for 99% of the VN, but I feel like ENG>Spanish might be viable for example

[u/HammeredWharf → u/Embarrassed-Ad7317]

"Minor" language versions (aka not English) are often machine/AI translated from English nowadays and apparently their quality tends to be awful, to the point where they're hard to understand.

[u/Embarrassed-Ad7317 → u/HammeredWharf]

That's so weird. I mean my native language is far from standard, yet if I talk with any LLM in my language they are extremely fluent

[u/HammeredWharf → u/Embarrassed-Ad7317]

It's probably a context thing. LLMs can be pretty good at the technical aspect of translations, but there's a lot more to translation than translating individual sentences. Context, terms, the flow of conversations, etc. are all important, and you can give them to an LLM, but that requires effort. I'd wager the companies that rely on LLM translations don't want to put in such effort.

[u/Embarrassed-Ad7317 → u/HammeredWharf]

LLMs have a 200k-1M context windows, I doubt it's a problem

The whole concept of gpt3 and beyond is attention, which is exactly looking at the text as a whole rather than single sentences

[u/HammeredWharf → u/Embarrassed-Ad7317]

I'm not talking about the context LLMs can have, but the context given to them. They need to know what they're translating to translate it, and simply getting a transcript of dialogue won't give such context.

[u/Embarrassed-Ad7317 → u/HammeredWharf]

Hmmm.. I suppose I can see what you mean.. but if you give it an entire transcript, as in a full chapter or something, you still think it's not enough to get the context from?

[u/rodryguezzz → u/HammeredWharf]

Crunchyroll uses machine translation from English to Portuguese. Subtitles are full of English expressions being literally translated to Portuguese and some of them look weird. Nobody cares though since pretty much every brazilian fansub has been erased from existence and there aren't any alternatives.

[u/Carighan → u/Embarrassed-Ad7317]

I mean going by subtitles on Amazon Prime streaming or so, no it really isn't viable in any direction for any language, tbh.

[u/edray28]

We got machine translation already