Translating to BASE64 is a good test to see how well it works as a language translator without changing things, because its the same skill for an AI model.
If the model changes things it means it didn't really capture the translation patterns for BASE64, so then who knows what it will miss when translating between languages if it can't even do BASE64?