Saving UTF-8 characters in the UI/interface#758

Right now, whenever you have UTF-8 chars in the XML/interface, once upon a time it brokes some of them. Not all and not every time, but when save and load multiple times, it breaks for example one or two characters.

I had a problem with those characters but expect also others :
ěščřžžýáííé (Czech characters)

You can see on the attached image, that the last text is broken but the previous one not. They should have the same character and they were scripted at the same time.

Půlčík
Trpaslík

The game supports UTF-8 very well. Only JSON feature is broken.
I wrote a guide (sadly, in Russian) how to fix it for a Russian mod:
https://steamcommunity.com/sharedfiles/filedetails/?id=2075879110

It’s not only for JSON. I am right now saving and loading whole XML and it’s also causing the problem. But maybe that’s because TTS using internaly JSON to save it. I will try you guide with Google translate, thanks!

Example

function onSave()
  return self.UI.getXml()
end

function onLoad(saved_data)
    self.UI.setXml(saved_data)
end

I think I found a simmilar issue using getXml and setXml even without UTF-8 characters, (I stripped them to see what is going on.)

I get an exception which is not caught using pcall.

Error in Script (Global) function

<startLuaCoroutine/on_next_frame_coroutine>: The output bute buffer is too small to contain the encoded data encoding ‘Unicode (UTF-8)’ fallback System.Text.EncodeReplacementFallbacl’. Parameter name bytes.

Just want to add that I’m experincing this also, trying to add Chinese characters. I finally witnessed it happening. My Atom code was looking fine, then after saving (Ctrl+shift+S), the same text updates to have the “diamond ?”. My guess is the conversion to or from JSON is doing it. Unfortunately it feels a bit random, as I can have multiple copies of the same chineses phrase, yet only one of them will get the diamond?