Is DeepL any better than Google?

[sofiadparamo]

I created my own implementation for using DeepL API because I saw many people point out that this service is way better and more natural than Google's translation, however, when I tried to use it in a particular use case (In my case, dubbing videos from Spanish to other languages) and me being bilingual myself, I noticed a lot more problems with the DeepL translation.

For instance:

Source	String
Original (Spanish)	"Que un juego para niños tenga como logro estafar a un pobre aldeano es bastante curioso"
Translated by me	"The fact that a game for children has an achievement for scamming a poor villager is pretty curious"
Google Translate	"That a game for children has the achievement of swindling a poor villager is quite curious"
DeepL	"That a children's game should be able to rip off a poor villager is quite curious"

The DeepL result is just a plain wrong translation. I also got some wrong translations from Google, but they're overall more coherent, at least to my appearance.

What are your thoughts on this? Have you experienced better results using DeepL in other projects?

[qgustavor]

I got curious about those translations and experimented comparing those to those of some models available in Hugging Face:

Model	Downloads last month	Translation
Helsinki-NLP/opus-mt-es-en	270,778	That a game for children has as its goal to con a poor villager is quite curious
salesken/translation-spanish-and-portuguese-to-english	11,447	It is rather curious that a child's game has the effect of stabbing a poor villager.
Helsinki-NLP/opus-mt-tc-big-cat_oci_spa-en	55	That a game for children has as an achievement to defraud a poor villager is quite curious

Weirdly seems the model with less downloads got the best translation. Maybe because it's a big model, so maybe it uses more resources in exchange to get better translations I think.

I wonder if is the word order (English Wikipedia, Spanish Wikipedia) from the original phase that makes it hard for machine translation. I don't speak Spanish, I studied it a bit but I don't speak, nevertheless this order seems a bit weird to me. Maybe the training data those models had relatively few phases in this order. If you rewrite it as "Es bastante curioso que un juego para niños tenga como logro estafar a un pobre aldeano" - which is a order that for some reason feels more natural to me - then you get the following translations:

Model	Translation
Google Translate	It is quite curious that a game for children has the achievement of swindling a poor villager
DeepL	It is quite curious that a children's game has the achievement of swindling a poor villager.
Helsinki-NLP/opus-mt-es-en	It's quite curious that a game for children has as its goal to con a poor villager.
salesken/translation-spanish-and-portuguese-to-english	It is rather curious that a children's game has the effect of stabbing a poor villager.
Helsinki-NLP/opus-mt-tc-big-cat_oci_spa-en	It is quite curious that a game for children has as an achievement to defraud a poor villager

Now seems DeepL performs a lot better (while Helsinki-NLP/opus-mt-es-en still performs badly). Probably DeepL needs to improve their training set to avoid this kind of issue.

[sofiadparamo]

Wow, that's super interesting, I totally missed the fact that the phrase in Spanish was weirdly ordered, even though I fixed that in my own translation. Rephrasing the subtitles manually prior to translation should improve the results a lot! Thanks!