Give Feedback
0 of 0
File information
Created by
trawzifiedUploaded by
trawzifiedVirus scan
Some manually verified files
About this mod
Patch Skyrim player dialogue through the power of AI! DialogueTransformer is a Synthesis patcher that can convert player dialogue using a text2text large language model (LLM).
- Requirements
- Permissions and credits
- Changelogs
- Donations
DialogueTransformer is a Synthesis patcher that can convert player dialogue using a text2text large language model (LLM).
Through these models, the patcher can dynamically change dialogue to fit certain patterns. In short; with this patcher, you can easily overhaul player dialogue to fit certain manners of speech to give your character more personality! Models are available for download separately due to their file size, they are listed below.
The following models are available for download right now. Many more coming soon!
- Khajiit Speak
- UwU Speak
1. Install Synthesis.
MAKE SURE TO INSTALL THE LATEST PRE-RELEASE VERSION (PR004) FROM HERE, OTHERWISE THE SETTINGS WON'T SHOW!
This is where you'll need to start the patcher from to generate a plugin that will change your dialogue.
Installed a new mod? Make sure to re-run the patcher to convert newly added dialogue.
2. Manually download the main synth file from the file section and open it, this will add the patcher to Synthesis.
Make sure to open Synthesis at least once, otherwise your PC won't recognize the file type.
Alternatively, you can also install DialogueTransformer from the Git section in Synthesis itself.
I also recommend setting the Mutagen and Synthesis version to 'Match' (image).
3. Install the DialogueInferencingClient in the optional files through your mod manager. This is required in order for the patcher to use the LLM to inference dialogue from. In case you forget this step, the patcher will try to manually download the client, but I recommend installing it beforehand to avoid issues caused by connection drop-outs or anything like that.
4. Nearly there! Install whatever model(s) you would like to use through your mod manager, and select the model to apply in the Synthesis settings (image).
5. All set! Click the run button in Synthesis to generate a plugin (esp) that should patch all the dialogue for your setup (image).
DialogueTransformer is a Synthesis patcher that can convert player dialogue using a text2text large language model (LLM).
Through these models, the patcher can dynamically change dialogue to fit certain patterns. In short; with this patcher, you can easily overhaul player dialogue to fit certain manners of speech to give your character more personality! Models are available for download separately due to their file size, they are listed below.
The following models are available for download right now. Many more coming soon!
- Khajiit Speak
- UwU Speak
1. Install Synthesis.
MAKE SURE TO INSTALL THE LATEST PRE-RELEASE VERSION (PR004) FROM HERE, OTHERWISE THE SETTINGS WON'T SHOW!
This is where you'll need to start the patcher from to generate a plugin that will change your dialogue.
Installed a new mod? Make sure to re-run the patcher to convert newly added dialogue.
2. Manually download the main synth file from the file section and open it, this will add the patcher to Synthesis.
Make sure to open Synthesis at least once, otherwise your PC won't recognize the file type.
Alternatively, you can also install DialogueTransformer from the Git section in Synthesis itself.
I also recommend setting the Mutagen and Synthesis version to 'Match' (image).
3. Install the DialogueInferencingClient in the optional files through your mod manager. This is required in order for the patcher to use the LLM to inference dialogue from. In case you forget this step, the patcher will try to manually download the client, but I recommend installing it beforehand to avoid issues caused by connection drop-outs or anything like that.
4. Nearly there! Install whatever model(s) you would like to use through your mod manager, and select the model to apply in the Synthesis settings (image).
5. All set! Click the run button in Synthesis to generate a plugin (esp) that should patch all the dialogue for your setup (image).
WARNING: The first time run of this patcher may be quite slow! Depending on your hardware and the amount of dialogue that needs to be inferenced, it may take a while. The patcher will list how long it will take before it's done in the log. After the first run, the model will store the cache of previously converted records and use this, so the second time it will re-use these and complete within a few seconds.
Want to help me improve the speed of the patcher? Send me the _LocalCache.csv file that was generated after running the patcher, and I will add it to the pre-generated cache for everyone to use!
First, I wanted to make a translation patcher. Every bit in Skyrim, translate it from English to German for example, to make Skyrim modding more accessible to everyone. But that was a bit too large in scope, so then I wanted to make a Khajiit Speak patcher because it's a huge hassle to set that up. You almost have to build your load order around the mods that have Khajiit Speak patches available to them, when it would be much easier if you just ran a patcher over those mods and called it a day. It's also super easy to just miss a patch somewhere, when there are so many out there.
So then I started writing some code to apply those little bits of Khajiit speech 'this one', 'warm sands' to replace words in the normal Skyrim dialogue. That worked, but the end result was just kind of... predictable? It wasn't really close to the level of detail that a normal human being could come up with, just simple word replacements here and there. I also had trouble converting the nouns into the third person, there's tons of those and I couldn't really find a library or resource to get those from. At the same time, I was experimenting with ChatGPT to see if it could write bits of my patcher code. That's where I got the idea; what if I could use ChatGPT to translate those dialogue bits to Khajiit speech?
Now, that's where I dove into the world of machine learning, and I found out that there's lots of decent Text2Text generation models out there like T5. I found out you can fine-tune those models by providing the model with extra data, so I downloaded as many Khajiit speech patch mods as I could and wrote a small program that extracts all of the base dialogue along with the Khajiiti conversions. I fine-tuned a model on that, took me about two weeks to figure out how all that worked in Python, and there it was at last; a fine-tuned model that could convert any dialogue into Khajiit speech!
Small problem; how am I going to generate a Skyrim plug-in with these conversions in Python? There's no Synthesis or Mutagen equivalent for that, and Python is also kind of slow. All my attempts at loading the generated model in C# failed miserably, setting up the tokenizer and everything did not work out. So I did some more research and found out I can compile the Python script I use (using pyinstaller) to inference the model to a regular executable, that way I can start it up in C# and communicate with it (IPC) to fetch me the dialogue I want. This is what the DialogueInferencingClient is.
Now onto the next problem, which was performance. Sure, the patcher worked, but it's just not very fast, it's very expensive to inference the LLM. On my system (7900XTX, 5950X and 32GB of 3600MHz DDR4 RAM) it took a little less than a second for one average line of dialogue. That may not seem like much, but consider a modded Skyrim setup will likely have thousands of lines, the biggest load orders can have like 30000 lines it has to convert. That would take over 8 hours to do, and my system is quite powerful already. I did some testing with a friend that had an NVIDIA GPU (3080Ti), so we could try CUDA, run the model on the GPU. It was about three times as fast, doing 3.30it/s. A lot better, but it was quite a hassle to set up, and I don't want to require everyone to go through such a painful process so the patcher can finish in a reasonable amount of time.
First I applied some multithreading optimizations. We can start up multiple DialogueInferencingClients depending on the power of the system, and allocate parts of the work that needed to be done to multiple threads. That about doubled the patching speed, going from 1.33it/s to 2.4it/s, so that's 'only' 4 hours now.
Four hours may be better than eight, but that's still too long, nobody wants to have their PC running at full power for half of the day generating Khajiit speech. Then I came up with another idea; we can apply caching here! A lot of the lines in everyones modded setups will be the same, they may have the same mods installed for example. If I can calculate those in advance (pre-caching), the patcher wouldn't need to do so much of the heavy lifting required. So I crowdsourced some exports of the dialogue people had in their modded Skyrim setups, booted up Linux so I could run the model on my GPU (please AMD, bring ROCm to Windows), and I generated pre-cache for all those bits. I ended up with nearly 62000 lines, and the patcher speed for people with cache went down to only about 10 seconds rather than a couple of hours!
Now of course I cannot assume I have everything cached, so it will still take a bit depending on your system and how much of the dialogue in your load order has been cached. If you're still reading this though even after all the technical talk, I would love for you to send me your _LocalCache.csv after you run the patcher. It will contain dialogue lines I don't have in the cache yet, so if you want to help me make the patcher work faster, you know what to do! I also want to try and create new and better models that can actually translate all of the text in Skyrim to other languages, but that might be a while off yet. I'm going to play Baldur's Gate now :)
- Gregarious Jamie & Aljo for helping me test the patcher
- Aljo, Althro, Cacophony, Ceadeus, Charlistic, Chef, Codygits, ForgottenGlory, Gaxos, GreatPadinski, Gregarious Jamie, I unpause, IAmMe, Lively, Moose, Slim, TwistedModding, Ugo and Zyian for helping me crowdsource dialogue to generate model pre-cache.
- Aljo, Althro, Cacophony, Ceadeus, Charlistic, Chef, Codygits, ForgottenGlory, Gaxos, GreatPadinski, Gregarious Jamie, I unpause, IAmMe, Lively, Moose, Slim, TwistedModding, Ugo and Zyian for helping me crowdsource dialogue to generate model pre-cache.
- Noggog for creating Synthesis/Mutagen and always helping me out in the Synthesis Discord.
- Koveich for creating the awesome Sovngarde font I'm using in the headers here.
- Koveich for creating the awesome Sovngarde font I'm using in the headers here.