NEW: Check out xVATrainer, which you can now use to train your own voices for xVASynth: https://www.nexusmods.com/skyrimspecialedition/mods/65022
Check the description for the current voices released, and the linked google doc for all voices from all games, from both the community (xVATrainer) and myself.
Join and check the Discord, or on Patreon, what I'm currently working on next. I also maintain an FAQ on discord: https://discord.gg/nv7c6E2TzV (the quickest method to reach me)
When i am trying to generate voices, using batch generation, i got this HOW THIS CAN BE FIXED???
Traceback (most recent call last): File "server.py", line 335, in do_POST File "python\xvapitch\model.py", line 326, in infer_batch File "D:\Modding\XVASynth\.\resources\app\python\xvapitch\xvapitch_model.py", line 223, in infer_advanced return self.infer_using_vals(logger, plugin_manager, cleaned_text, text, lang_embs, speaker_embs, pace, None, None, None, None, None, None, pitch_amp=pitch_amp) File "D:\Modding\XVASynth\.\resources\app\python\xvapitch\xvapitch_model.py", line 292, in infer_using_vals x, x_emb, x_mask = self.text_encoder(input_symbols, x_lengths, lang_emb=None, stats=False, lang_emb_full=lang_emb_full) File "torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "D:\Modding\XVASynth\.\resources\app\python\xvapitch\xvapitch_model.py", line 602, in forward x = torch.cat((x_emb, lang_emb_full), dim=-1) RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 27 but got size 1 for tensor number 1 in the list.
I just spent a couple days trying to get this to work, had some successes, and thought I'd share what I found in case this might help someone else out. It's pretty cool in terms of having a TTS synthesizer with tons of different familiar voices. I can finally make that DAZ animation of Strong's makeup roleplay ASMR video. STRONG DO YOUR MAKEUP! STRONG TELL YOU RELAX!
If you're stuck at the Loading Python back-end... dialog, that means that the resources/app/cpython_cpu/server.exe program isn't starting. Open up a command prompt, CD to that directory, and try running it manually to see why it's failing. I got a message that it couldn't find api-ms-win-core-path-l1-1-0.dll which turned out to be the same problem a bunch of newer blender python users are having. I replaced that dll and it fixed the Loading Python problem.
The next issue was a "Something went wrong" dialog when I tried to do some actual synthesis. My workstation is offline only. I could load a voice (that I had manually installed in models) but when I clicked the Generate button I would get a "Something went wrong" message. I saw a bunch of messages in the console log (Ctrl-Shift-I) about resources/app/javascript/nexus.js trying to connect to wss://sso.nexusmods.com/ over and over and thought that maybe it was trying to download something it needed. I tried a bunch of stuff including intercepting that and replaying a fake nexus login using a local lightweight web server to try to find out what it was to no avail. Finally I backed off to 3.0.0 and Generate Voice worked! If you're offline only, don't 3.0.2.
I couldn't get speech2speech to work due to what I think is a race condition or a directory issue, but if I record a wave file and then drop it onto the microphone, that works. The text voices are quite a bit higher quality though, so it's kind of moot. I can't get the voice delta or starting with a base and making a synth with my own voice to work. It sits on "Synthesizing" forever but there are messages in the console log about localhost:8008/simplesynthesis failing. I'll probably throw in the towel and take a look at xVATrainer for the stab at new voices. In any event, it was totally worth digging into this! This is an awesome TTS system that I'm already having a ton of fun with. Finally my silly DAZ animations will have some decent voices in them. Thankee kindly for all your hard work and AI expertise Mr. Ruta!
The tool is really good but I hope that in the future it will also work with other languages. Also in terms of Starfield, to give the player char a voice in the language in which you play.
I have a huge problem with version 3.0.0 that may or may not be just me so I'm asking if anyone else is or has experienced this issue.
After I copy and paste a line of dialogue and then click generate for the voice model, I can not make any difference in 'pitch' when adjusting the slider up or down, it makes no difference. Sure the slider moves and stays where I put it, but audibly it stays the same and doesn't change.
Has anyone else had this happen and how do I fix it? Re-installing makes no difference.
Give Feedback
284 comments
Check the description for the current voices released, and the linked google doc for all voices from all games, from both the community (xVATrainer) and myself.
Join and check the Discord, or on Patreon, what I'm currently working on next. I also maintain an FAQ on discord: https://discord.gg/nv7c6E2TzV (the quickest method to reach me)
HOW THIS CAN BE FIXED???
Traceback (most recent call last):
File "server.py", line 335, in do_POST
File "python\xvapitch\model.py", line 326, in infer_batch
File "D:\Modding\XVASynth\.\resources\app\python\xvapitch\xvapitch_model.py", line 223, in infer_advanced
return self.infer_using_vals(logger, plugin_manager, cleaned_text, text, lang_embs, speaker_embs, pace, None, None, None, None, None, None, pitch_amp=pitch_amp)
File "D:\Modding\XVASynth\.\resources\app\python\xvapitch\xvapitch_model.py", line 292, in infer_using_vals
x, x_emb, x_mask = self.text_encoder(input_symbols, x_lengths, lang_emb=None, stats=False, lang_emb_full=lang_emb_full)
File "torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "D:\Modding\XVASynth\.\resources\app\python\xvapitch\xvapitch_model.py", line 602, in forward
x = torch.cat((x_emb, lang_emb_full), dim=-1)
RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 27 but got size 1 for tensor number 1 in the list.
If you're stuck at the Loading Python back-end... dialog, that means that the resources/app/cpython_cpu/server.exe program isn't starting. Open up
a command prompt, CD to that directory, and try running it manually to see why it's failing. I got a message that it couldn't find api-ms-win-core-path-l1-1-0.dll which turned out to be the same problem a bunch of newer blender python users are having. I replaced that dll and it fixed the Loading Python problem.
The next issue was a "Something went wrong" dialog when I tried to do some actual synthesis. My workstation is offline only. I could load a voice (that I had manually installed in models) but when I clicked the Generate button I would get a "Something went wrong" message. I saw a bunch of messages in the console log (Ctrl-Shift-I) about resources/app/javascript/nexus.js trying to connect to wss://sso.nexusmods.com/ over and over and thought that maybe it was trying to download something it needed. I tried a bunch of stuff including intercepting that and replaying a fake nexus login using a local lightweight web server to try to find out what it was to no avail. Finally I backed off to 3.0.0 and Generate Voice worked! If you're offline only, don't 3.0.2.
I couldn't get speech2speech to work due to what I think is a race condition or a directory issue, but if I record a wave file and then drop it onto the microphone, that works. The text voices are quite a bit higher quality though, so it's kind of moot. I can't get the voice delta or starting with a base and making a synth with my own voice to work. It sits on "Synthesizing" forever but there are messages in the console log about localhost:8008/simplesynthesis failing. I'll probably throw in the towel and take a look at xVATrainer for the stab at new voices. In any event, it was totally worth digging into this! This is an awesome TTS system that I'm already having a ton of fun with. Finally my silly DAZ animations will have some decent voices in them. Thankee kindly for all your hard work and AI expertise Mr. Ruta!
Can someone link a tutorial in how to make a patch for a quest mod? I'm googling but finding nothing.
Thanks.
After I copy and paste a line of dialogue and then click generate for the voice model, I can not make any difference in 'pitch' when adjusting the slider up or down, it makes no difference. Sure the slider moves and stays where I put it, but audibly it stays the same and doesn't change.
Has anyone else had this happen and how do I fix it? Re-installing makes no difference.