Fallout 76

File information

Last updated

Original upload

Created by

Dan Ruta

Uploaded by

DanRuta

Virus scan

Some manually verified files

Tags for this mod

About this mod

AI tool for high quality voice acting synthesis using in-game voices from video games.

Permissions and credits
Donations
Quick intro

xVASynth is an AI based app for creating new voice lines using neural speech synthesis. The app loads models individually trained on voices from several games. The app gives users control over details such as pitch and durations of individual letters to provide control over emotion and emphasis. To see it in action, watch these short intro/tutorial videos, narrated by various supported voices:

  


Supported games

Discord: https://discord.gg/nv7c6E2TzV
Patreon: https://www.patreon.com/xvasynth
Twitter: @dan_ruta

Note: 
To keep things fair, avoid using the tool in an offensive/explicit manner. Make it obvious where you can in descriptions that the voice samples are generated, and not from the original voice actors. Any issues you cause with this are on you.

Note on cross-game use:  No game assets are uses/distributed by this tool. The game assets are only used during training as reference for the algorithm to drive itself towards a point where it can create voices close enough to the reference examples. The question of permissions is at the moment more a question about ethics rather than law, as nothing yet exists for neural speech synthesis.

Introduction

xVASynth (or [F76]VASynth, for [Fallout 76] voices) wraps around FastPitch [1] models trained on datasets compiled from in-game voice acted lines. The strengths of this model are in the artistic control over the generated audio. Once you generate the audio from your text prompt, you can adjust the pitch and durations using the editor:


The use of neural speech synthesis leads to natural sounding voices, something which is very difficult to do with more traditional methods involving concatenations of existing data. It also means new vocabulary can be generated, outside of what the voice actors have already read out.

For larger projects, where you need to synthesize a large amount of lines, you can alternatively use the Batch synthesis mode (v1.3.0+). You can use either a .txt file or a .csv file to batch generate hundreds or even thousands of lines, in one go, with parallelization. Although the pitch/duration editor is sometimes needed to get a line sounding just right, it's sometimes not needed, and this is a good way to get an initial pass on lines.

There are several potential use cases for this tool:
  • Creating voice lines for new quest mods
  • Creating machinima
  • Give voices to custom follower mods, or expand existing followers' with more life
  • Expand/edit/fix existing quests' stories/dialogue options
  • Add variety to in-game voices by adding packs from other games' voice (eg add Fallout/Oblivion voices to skyrim voice lists)
  • Add new vocabulary to voices not already found in scripts (cannot easily be done with audio splicing)
  • (Fallout 4) Add new player names to name lists
  • (Fallout 4) Give Nate/Nora a different voice (using the batch synthesis mode)
  • Enhance vanilla quests by adding more lore/explanations in conversations
  • Make English translations for mods voiced in other languages
  • (Morrowind) Add voice to unvoiced areas of the game
  • Change the voice used by a character (if you believe a different voice would suit them better)
  • memes
  • probably more...


Installation

You may need to install Microsoft Visual C++ Redistributable if you don't already have it.

Follow these steps:
  1. Download either the CPU-only or the GPU+CPU version (the GPU one just lets you generate audio faster, if you have an NVIDIA card and CUDA)
  2. Extract this anywhere (it is not tied to the game files)
  3. Download the Vocoder models (WaveGlow). Place these .pt files in the "<.exe location>/resources/app/models" folder. You do not need to update/redownload these files with different versions of the xVA app, unless otherwise specified.
  4. Download individual voice models, and place the three (or four) files into the  "<.exe location>/resources/app/models/<game>" folder. The folder structure in the download is correct, you can just merge it into the app folders like you would installing a texture mod manually
  5. Launch the app by double clicking the xVASynth.exe file as admin (you may want to create a shortcut to this somewhere convenient)

To confirm, when installing voices, you should see 3 files (a .json, a .pt, and a .wav file) all found in <your xVASynth install directory>/resources/app/models/<game>/   (where <game> is fallout76, for models on this page).

Important: Make sure you click "Allow" if windows asks you for permission to run the python server. I use a local HTTP server to enable communication between the python code (for the AI models) and the JavaScript code (for the Electron front-end). If there are any issues, check the server.log file (located next to xVASynth.exe) - there should be an error at the end which I'll need to see for helping with issues.

Check the faq channel on Discord if you run into any issues, or send me a message on the bugs channel if your issue is not fixed by its contents.

Follow this video for ffmpeg installation, if you decide to use audio post-processing on output.

For Fallout 76, the voices trained so far are as follows ("Track" the mod for updates):
  • 🌮 🗲 RobotAssaultron
  • 🌮 🗲 Robot MrHandy
  • [New] 🌮 🗲 MrGutsy
  • [New] 🌮 🗲 MODUS

Models in 2 weeks early access on patreon: (more coming soon, as requests come in)

Where green represents good quality, yellow means ok quality (but might need a good deal of playing with the input to get something good), and red being currently unsuccessful. The 🌮 emoji represents voices that have the bespoke Tacotron2 pre-processing step. This means the sentence composition quality will be high, and you won't need to mess around with the spelling too much, to get it to say what you want. The 🗲 emoji represents the availability of bespoke HiFi-GAN vocoder for a voice. This shows up in the vocoders dropdown in the app, as another option alongside the WaveGlow and quick-and-dirty vocoders. This means the actual audio quality will be very good (better than WaveGlow), while running much faster than WaveGlow (at the speed of the quick-and-dirty model).

For other voices to be added to this list, there need to be at least ~250 lines of dialogue in the game data (less may work but the quality will be lower). Usually, the more data, the better the quality. A transcript also helps quality, but it is not a hard requirement. The order of voices being released (across all supported games) is determined on the discord server/patreon page.

Tips

  • If you have an NVIDIA GPU, you can enable GPU inference for much faster speeds. You do need to install CUDA dependencies yourself first, as a pre-requisite (tested for version 10.1, check with "nvcc -V" in cmd to verify correct installation)
  • If you are using only the CPU to do inference, tick the "Quick and dirty" checkbox whenever you can (except for the final generation of the final audio). This option is much quicker, as it uses HiFi-GAN[2] instead of WaveGlow[3] for audio generation, but the quality is lower. Leaving it ticked on also speeds up the app start-up.
  • Aim to generate audio between 1 and 5 seconds long. Audio much longer than this starts breaking down in quality. If you have a really long sentence, try to break it down into separate clauses and splice the audios together in a tool like Audacity
  • You can right-click voices in the panel on the left to hear a quick preview of the voice


Downstream uses
If you make anything with this tool (mod or otherwise), let me know and I will include it here.

 


YouTube playlist of xVA experiments (WaveGlow MaleSlyCynical): https://www.youtube.com/playlist?list=PLDGgH-fuVvfa8-HFdSi7ls1ykLIuquIpD

[Fallout 4] Subversion - The Institute-Railroad Alliance Alternate Endinghttps://www.nexusmods.com/fallout4/mods/50975
An immersive alternate Railroad and Institute ending that allows the reformation of the Institute and peace between the Commonwealth factions. Free the synths, end the kidnappings, stop the immoral experiments, pacify the Brotherhood of Steel with less violence. 4 new companions are available: Z1-14, Glory, Super Mutant Virgil, Synth Elder Maxson.

[Fallout 4] Fallout 4 Office Drama Simulator - An Immersive Pre-War Roleplaying Mod
https://www.nexusmods.com/fallout4/mods/51127
Three years before the Great War broke out, before little baby Shaun was ever conceived, his parents Nate and Nora were briefly employed in a promising tech company founded by one of Preston Garvey's ancestors, Tyrone Garvey. But it wasn't always a smooth ride in the cutthroat world of corporate politics. Enjoy this pre-War roleplaying mod / TC.

[Fallout 4] Marked for Termination - A Terminator-Inspired Manhunt in the Commonwealth
https://www.nexusmods.com/fallout4/mods/31356
In the future the Sole Survivor of Vault 111, you, will lead mankind in a war against a malevolent artificial intelligence programmed to enslave the world. From this dark future two machines have been sent back, one to kill you, the other to protect you. Live or die... war never changes.

[Skyrim] Phenderix Magic World: https://www.nexusmods.com/skyrimspecialedition/mods/6551
Phenderix Magic World adds a massive amount of content including new spells, weapons, bosses, followers, locations, and much more! Download today to unleash hundreds of new roleplaying options and discover a new world where magic has just begun to awaken.

[Fallout 4] Nuka-World Rebornhttps://www.nexusmods.com/fallout4/mods/32857?
Nuka-World Reborn is a quest mod which not only allows you to have multiple options to get rid of the raiders, but it adds new questlines to Nuka-World and allows you to play as a Trader.

[Skyrim] Positive Undressed Reactions: https://www.nexusmods.com/skyrimspecialedition/mods/44334
New lore-friendly voiced reactions to the player being undressed, using xVASynth, unused lines and splicing.

[Skyrim] The Windhelm Smelterworks: https://www.nexusmods.com/skyrimspecialedition/mods/46333
The Windhelm Smelterworks adds an industrial scale smelterworks outside Windhelm, bringing some depth and immersiveness to Skyrim’s heavily mining dependent economy

[Skyrim] Cait in Skyrimhttps://www.nexusmods.com/skyrimspecialedition/mods/44982
Bring Cait from Fallout 4 to Skyrim. Fully voiced using Cait's voice! Currently has 79 voiced lines. I've recreated a number of the original lines from Fallout, some "tweaked" ;) and a LOT of new lines!

[Skyrim] The Elder Scrolls Legends Importshttps://www.nexusmods.com/skyrimspecialedition/mods/32104
Adds references to Elder Scrolls: Legends cards and story into the world of Skyrim.

[Oblivion] NewCity_SI_Passwall_ENG: https://www.nexusmods.com/oblivion/mods/50837
Here you have the Russian mod NewCity Passwall, fully translated into English

[Skyrim] Ysolda Roasts Jonhttps://www.nexusmods.com/skyrimspecialedition/mods/44630
Recreates the famous "Lamar Roasts Franklin" scene with a skyrim flavor using Ysolda and Jon Battle Born

[Skyrim] Dyudyaev-Kun: https://www.nexusmods.com/skyrimspecialedition/mods/44753
Adds a Male Woodelf dragonborn to your game.
He is a custom voice follower based on the Male young eager voice. ( use SKVA Synth)

[Skyrim] Authentic Sinding Follower SE: https://www.nexusmods.com/skyrimspecialedition/mods/33517
Gives Sinding a whole visual makeover and makes him a potential follower if you decided to help him during the Daedric quest "Ill Met by Moonlight".

[Skyrim] Random Guard Dialogueshttps://www.nexusmods.com/skyrimspecialedition/mods/44378
New funny and random voiced dialogues for guards made with xVA-Synth

[Skyrim] Stop right there criminal scum
https://www.nexusmods.com/skyrimspecialedition/mods/44181
A mod to add the infamous Oblivion line to Skyrim guards

[Skyrim] Trigger King Olaf's Festival Any Day - With Proper Ending SE
: https://www.nexusmods.com/skyrimspecialedition/mods/46766
With this mod you can tell Viarmo to spontaneously arrange a festival on the same night. Also makes sure that each festival will automatically end at 4AM

[Skyrim] Your Choices Matter - A Dark Brotherhood Expansion
: https://www.nexusmods.com/skyrimspecialedition/mods/46871 (LE)
This mod extends the Dark Brotherhood questline in many ways and adds an optional alternate ending. The ending you get will depend on the choices you, the Player make, throughout the story. Completely voiced dialogues, using vanilla and xVASynth assets.

[Skyrim] Adoption without Murder (Innocence Lost for Good Guys): https://www.nexusmods.com/skyrimspecialedition/mods/46741
A tongue-in-cheek alternative solution for Innocence Lost, for when you maybe want to adopt an orphan but without all that murder and stuff.


Future Plans

Generally the plan is to keep going down the fairly long list of voices remaining, across all supported games. I do plan on returning to some the voices already released to improve them with further/re-training.

There are quite a few voices left to train (across all games). You can track/vote on further progress of the models being trained on my patreon page (which I'm hoping will help fund the new hardware, so I can make more/better models), or on the GitHub page, at https://github.com/DanRuta/xVA-Synth.

Support

The best support is using the tool, making something cool with it, and letting me know about it! Or spreading the word, to anyone that may get some use/fun out of this. Join the discord here, and let me know if you have any ideas/suggestions, show off something you made, or you just want to chat about this: https://discord.gg/nv7c6E2TzV

Special thanks:

Hector Medima, CinnaMewRoll, Grant Spielbusch, Rachel Wiles, Max Loef, CHASE MCKELVY, Jay Gee, Matthew Warren, Mark , Loni Dennis, Alan Gonzalez, SpaceD0lphin, Jonathon Barton, Nathan J. Goode, lord parker, Tom Green, PConD, Cutup, J , Joseph Paul Dennison, Krazon, Tara Cooksey, George Osgerby, Caro Tuts, PlayerDeer , Blythe, Snud Swimp, Tako-kun, Retlaw83, Sh1tMagnet, Yael van Dok, PorcelainShrine , Passionate Lobster, Drake , Lady Steel, Ashley Higgins, FinalFrog, Donald Bass, Hazel Louise Steele, J. Quint, Lulzar , Vahzah Vulom, batteryjar , Five More Minutes, Ryan W, Laura Almeida, Alexandra Whitton, Zelda Hadley, Cookie , Baki Balcioglu, Beto, Harsh , Pseudo Immortal, My Best Friend Is A Squid, Agito Rivers, Thuggysmurf, radbeetle 

All the amazing donors, anonymous or otherwise.
Adrian Łańcucki for FastPitch and the helpful discussions on GitHub.
All the amazing researchers behind the many tools and models I've used in creating this.


References
     [1] FastPitch - https://arxiv.org/abs/2006.06873
     [2] HiFi GAN - https://arxiv.org/abs/2010.05646
     [3] WaveGlow - https://arxiv.org/abs/1811.00002
     [4] Tacotron2 - https://arxiv.org/abs/1712.05884
     [5] LJSpeech - https://keithito.com/LJ-Speech-Dataset/

Changelog:

v1.3.4 hotfix
- Improved error handling for when models cannot be found by the app (caused by changing models location to somewhere where there are no models, or moving the app installation without first updating the models path in the settings)

v1.3.3 hotfix
- Fixed occasional issues with loading models from certain custom paths

v1.3.2 
- Added batched batch mode - so parallelizing voice generation (limited only by your VRAM)
- Option to skip batch generating lines which have already been generated
- Globally improved voice generation quality, where word spacing was too short (zero) by default
- Various UI tweaks
- Speed optimizations for the batch synthesis mode

v1.3.1 
- Added settings to use per-game custom (user specified) models directories
- Made the model loading automatically refresh with file changes (re-starting the app no longer needed when installing new models)
- Various quality-of-life UI tweaks

v1.3.0 
- Added batch synthesis mode! (From a .csv or .txt file)
- Added partial re-generation! For changing only parts of the text without losing editor progress
- Fixed silence padding ffmpeg setting no longer working
- Added an option to keep the same pace value on new text line generations (on by default)
- Fixed the reset letter button not taking the sequence pacing into account
- Added option to automatically switch to HiFi-GAN vocoders when loading a voice, if one is available (on by default)
- Made the UI snappier (reduced the fade transition times)
- Fixed some settings options always defaulting to true, on app start-up
- Added even further resilience to ffmpeg not overwriting files
- Changed some things under the hood to enable non-multispeaker FastPitch support (easier for others to add their own models to the app)
- Added game support for: Cyberpunk 2077, Civilization
- UI tweaks
- Updated the Fallout 4 skin image to a higher res version

v1.2.3
- Fixed some issues with ffmpeg use when overwriting files
- Added support for a few new games (and game series): Fallout 76, Mass Effect, Half Life, The Witcher
- Re-implemented the game selection menu, to support a long list of other supported games
- Added app resilience against some extreme edge cases with badly formed models folder structure

v1.2.2
- Fixed bespoke HiFi models not showing up in the dropdown
- Changed output directory settings inputs to a single input field, for the currently loaded game
- Made clicking outside the editor de-select the letter(s)
- Fixed case where dragging a slider for a letter outside a selected group also moved it without first de-selecting the group
- Added a dark mode setting for the input text sequence, for accessibility

v1.2.1
- Fixed reset button not resetting pacing, and the "Reset Letter" doing it (swapped)
- Fixed voice-specific sub-folders not being automatically created in the output directory

v1.2.0 
- Added support for bespoke voice vocoder models - a fourth file in voice packs, a HiFi GAN model trained specifically for that voice with higher quality than either WaveGlow vocoders, at the speed of the "quick-and-dirty" model. Denoted by a "🗲" symbol next to the dropdown where available (To start with just F4: Nate)
- Added support for models saved as FP16, meaning models from now on will only be half as big (in terms of file size)
- Added an option to select which Vocoder you prefer to use (following feedback on v1.1 vocoder model change) - useful for deciding which to use for the best quality
- Added an amplitude multiplier ffmpeg audio post-processing option
- Added a search bar for the models selection panel
- Fixed issue where keyboard shortcuts were active when typing in input fields
- Added a specific error message to when the output directory for saving audio files does not exist
- UI tweaks/changes, enlarged minimum size of modals, for extremely small window sizes (the window can be resized)
- Fixed some occasional issues with file re-naming (and then editing)
- Made the "Enter" key act as a submit action when in the rename modal (and other prompt modals)

v1.1.1
- Various bug fixes on v1.1 features from feedback on the beta version
- Added letter length support to the multi-letter editing mode
- Enlarged the changelog/updates menu to fit the complete changelogs
- Made amp/flat incr/decr buttons work only on selected range when multiple letters are selected
- Fixed an issue with Amplify/flatten, increase/decrease buttons starting to overreact after the app has been in use for some time
- Fixed issues with sluggishness of the app after some time of use has passed

v1.1.0 
- Ability to edit previously generated samples
- Numerical inputs for sliders
- Upgraded WaveGlow to a better version. Bigger and a bit slower, but better sounding output across the board
- Re-implemented a big part of the letter lengths mechanism, fixing many issues in the process
- Ability to edit multiple letters at once (ctrl+click multiple), moving pitch slider together as a group
- Compiled using CUDA 11.0 (fix for 3000 series GPUs)
- Shift-click "Keep sample" to first bring up a file name prompt
- Stopped disabling the "Keep sample" button (useful for when outputting several audio post-processing versions)
- Made clicking on the letter bring focus to it (instead of clicking the slider and accidentally changing its value)
- Made the pitch editor taller
- Show the values for the editor sliders in a local tooltip
- Fixed bug with renaming files with explicit file format in the name
- Fix mp3 audio post-processing output sometimes failing
- Fixes to file re-naming if full stops were present in the input sequence
- Several other internal changes, including tweaks for any potential memory leaks
- Keyboard shortcuts, with a new "Information" menu, containing this cheat sheet:
- left/right arrows: Move between letter focused
- SHIFT-left/right: multi-letter create selection range
- up/down move pitch up/down for the letter(s) selected
- CTRL+left/right arrows change the sequence-wide pacing
- CTRL+up/down increase/decrease buttons
- CTRL+SHIFT-up/down amplify/flatten buttons
- space bring focus to the input textarea
- Enter: Generate sample
- CTRL-S: Keep sample
- CTRL-SHIFT-S: Keep sample (but with rename prompt)
- Escape: close modals
- Y/N for prompt modals

v1.0.9
- Added audio post-processing (via ffmpeg, you need to install it to enable this), to allow audio settings such as: file format, silence padding, sampling rate, and bit depth
- Bug fixes for the pitch/length editor, for when the input is changed (then "Something went wrong" happens)
- Reduced the rate of increase/decrease of the Amplify/Flatten buttons

v1.0.8
- Changed the default window size (you can always resize this by dragging the corner) and enlarged settings menu
- Fixed bad default custom filepaths in settings that may have lead to some audio output issues (you will still need to adjust this if you've already loaded v1.0.7, as those values are now bound to your personal settings - remove the double "resources/app" if you have that - just check that the path exists)

v1.0.7
- Made the amplify/flatten buttons additive instead of multiplicative (to avoid it getting out of hand)
- Fixed the letter length/pacing sliders! (I could never reproduce the issue, but I'd be surprised if this didn't fix it)
- Fixed a few small path issues, potentially also fixing a bug where temporary files were not getting cleaned
- Small UI changes, including file record hover effect, and including the file name in the deletion prompt
- Fixed newline characters in the prompt breaking file saving
- Added the ability to specify (on a per-game basis) what directory the output files should be placed in, when saved

v1.0.6 
- Added a file logger for the front-end to aid with user support/debugging - app.log
- Fixed audio not saving if filename-unsafe characters are in the input sequence
- Reduced freedom of range on the pitch slider to reduce bad quality output
- Fixed issue with auto-regen spamming the model with every slider tick and filling the screen with modals (all crashing if also making the app run out of vram)
- Fixed issue with deleting newly generated files
- Other small tweaks and fixes

v1.0.5
- Bug fixes

v1.0.4
- Bug fixes

v1.0.3
- Improved the UI - reorganized the buttons to group the letter-specific tasks to the top bar, and sequence-wide setting on the bottom bar
- Added a "Reset letter" button 
- Made it more clear which letter is being edited by highlighting it in red
- Added an option in the settings for auto-playing the audio when it's generated
- Fixed an issue where question marks in the input broke file saving

v1.0.2
- Fixed some filepath issues causing infinite loading

v1.0.1
- Added updates menu