We have released v0.31.0-alpha. It supports Anniversary Edition and includes a brand new optional speech recognition engine: Voice2Json to replace the old Microsoft speech recognition engine.
Note the engine is always optional. If you remove the Engine=Voice2Json line from the configuration file, the Microsoft Speech Recognition Engine will be used. Then you can use the alpha version like the beta version, and you don't need to install Docker for Windows.
The new engine will solve many problems: 1. Microsoft's speech recognition engine is bound to the system locale, making it difficult to make the engine work properly. 2. Lower recognition rate. You have to speak clearly like an announcer to be likely to be recognized. 3. The probability of false triggering is high. After a while, any noise the microphone picks up will trigger a random command.
The new engine does not have the aforementioned drawbacks. For English speakers, it has far better recognition accuracy than Microsoft and is less prone to false triggering.
However, the new engine is not perfect, and some non-English languages may not work at all (Known issues: Chinese and Russian works poorly). We look forward to your feedback.
Also, in order to run the new engine, you must have Docker Desktop installed. You will also need to modify the configuration file for the new engine to take effect (the default engine is still Microsoft).
dragonborn_speaks_naturally.dll contains code that hooks into SkyrimSE/VR.exe in order to take over dialogue and console commands. This hooking method is similar to a virus. Meanwhile, DragonbornSpeaksNaturally.exe has code to execute cmd commands in Windows in order to connect to the Voice2Json speech engine in docker. This makes some antivirus software think they are suspicious. In addition, DragonbornSpeaksNaturally.exe uses your microphone, which further increases the suspicion of antivirus software on it.
Therefore, every new version we upload will be quarantined and need to wait for the review of a NexusMods staff. This is why most of our releases are flagged for manually verified.
So are these files really doing the malicious things that antivirus software fears? I think the answer is no. dragonborn_speaks_naturally.dll just does what you want it to do: hooks into Skyrim, takes over Skyrim's dialogue and console commands, and simulates keypresses as you dictate. DragonbornSpeaksNaturally.exe also does what you ask it to do: record from your microphone, pass it to one of the speech recognition engines (Microsoft or Voice2Json), recognize the command you want to run and send it to dragonborn_speaks_naturally.dll. Then, dragonborn_speaks_naturally.dll will execute these commands (console, dialog or simulated keystrokes). During the entire process, your microphone recording never leaves your computer. And all the behavior of the mod depends on the configuration files you write and what you say through the microphone. No third party can remotely control its behavior.
The mod will not connect to the Internet while it is running (If you choose the Voice2Json engine, some model files will be downloaded from GitHub on the first launch, and once the download is complete, no internet connection is required; the Microsoft engine also does not use an internet connection). If you see an Internet connection in the VirusTotal behavior report, it's not the mod, but the VirusTotal's scanner are downloading the zip file from their own servers. The same page will also record the behavior of the scanner to decompress the DSN zip with 7z, which proves that not all the behaviors recorded are from the scanning target.
If you still have doubts about the safety of the mod, you can review its code yourself or ask experienced people. All of our releases are compiled from the following repositories, using Visual Studio Community 2022, to the x64 Release target: https://github.com/YihaoPeng/DragonbornSpeaksNaturally
Hey just want to say I love this mod, I'm wondering though, is there a version or way to use this with the dialogue interface option only? I loved being able to talk to NPCs and companions directly but I was very confused at first when my character started throwing gang signs (from me talking into my mic randomly and it switching weapons or active spells). It took me a while to realise it was from this and my mic 😅. I'm planning on doing a playthrough when I get ahold of the anniversary edition in the near future and I would love to give this another try. Just hoping I don't have to stay still and quiet while exploring or battling. Thanks!
Having an issue where the mod works sometimes, but not always. I dont know if it works only on conversations with few dialogue choices, but I have that suspicion. I feel like it works when there are only 3-4 choices, but never works when there are 10+.
I have a downgraded SSE and this mod keeps crashing my game, it always happens when I am in dialogue. No crashes when the mod is disabled. Load order crash log
If you have installed a DSN for AE, you will need to remove it from the mod manager and re-add it. Then you need to select the correct game version (Skyrim SE 1.5.97) in the DSN's installation dialog.
AE's dll does not work with SE 1.5.97 and will crash the game. Different selections in the installation dialog will cause different dlls to be released.
Note: We have released a version for AE 1.6.1170, which can be used if you need. (As before, because the virus scan issue is pending administrator review.)
When you launch Skyrim for the first time after installing **Docker for Windows**, DSN will automatically install **voice2json** with the **docker** command for you. When the installation is complete, you will see a `dsn_voice2json` container in the Docker Desktop UI.
This isn't the mod authors fault really its the new 1130 update on steam. Since it removed steam integration for the game things like achievement enablers and a lot of skse plugins just don't work with the update. I have heard that this issue will be fixed in the next update from steam but we shall see.
Can anybody help me limit the voice recognition commands to skyrim only or turn off all the ones that do things like open browsers and do keybinds? It's really annoying when I get lag spikes to find out my browser has been opened or I hear it constantly muting and unmuting my mic on discord. Thanks.
Are there steps to troubleshoot? Either for Alpha or Beta version. Log gives no errors. Beta MS engine shows mic activity. For alpha, docker is running and docker training command line calls work. Neither are selecting dialog (SkyrimVR has focus). Is there debug log to be able to see why speaking is not selecting dialog options or exit statements don’t exit dialog.
1122 comments
We have released v0.31.0-alpha. It supports Anniversary Edition and includes a brand new optional speech recognition engine: Voice2Json to replace the old Microsoft speech recognition engine.
The new engine will solve many problems:
1. Microsoft's speech recognition engine is bound to the system locale, making it difficult to make the engine work properly.
2. Lower recognition rate. You have to speak clearly like an announcer to be likely to be recognized.
3. The probability of false triggering is high. After a while, any noise the microphone picks up will trigger a random command.
The new engine does not have the aforementioned drawbacks. For English speakers, it has far better recognition accuracy than Microsoft and is less prone to false triggering.
However, the new engine is not perfect, and some non-English languages may not work at all (Known issues: Chinese and Russian works poorly). We look forward to your feedback.
Also, in order to run the new engine, you must have Docker Desktop installed. You will also need to modify the configuration file for the new engine to take effect (the default engine is still Microsoft).
The specific installation steps are here: https://github.com/YihaoPeng/DragonbornSpeaksNaturally/releases
---
Manually Verified / Quarantined Files
dragonborn_speaks_naturally.dll contains code that hooks into SkyrimSE/VR.exe in order to take over dialogue and console commands. This hooking method is similar to a virus. Meanwhile, DragonbornSpeaksNaturally.exe has code to execute cmd commands in Windows in order to connect to the Voice2Json speech engine in docker. This makes some antivirus software think they are suspicious. In addition, DragonbornSpeaksNaturally.exe uses your microphone, which further increases the suspicion of antivirus software on it.
Therefore, every new version we upload will be quarantined and need to wait for the review of a NexusMods staff. This is why most of our releases are flagged for manually verified.
So are these files really doing the malicious things that antivirus software fears? I think the answer is no. dragonborn_speaks_naturally.dll just does what you want it to do: hooks into Skyrim, takes over Skyrim's dialogue and console commands, and simulates keypresses as you dictate. DragonbornSpeaksNaturally.exe also does what you ask it to do: record from your microphone, pass it to one of the speech recognition engines (Microsoft or Voice2Json), recognize the command you want to run and send it to dragonborn_speaks_naturally.dll. Then, dragonborn_speaks_naturally.dll will execute these commands (console, dialog or simulated keystrokes). During the entire process, your microphone recording never leaves your computer. And all the behavior of the mod depends on the configuration files you write and what you say through the microphone. No third party can remotely control its behavior.
The mod will not connect to the Internet while it is running (If you choose the Voice2Json engine, some model files will be downloaded from GitHub on the first launch, and once the download is complete, no internet connection is required; the Microsoft engine also does not use an internet connection). If you see an Internet connection in the VirusTotal behavior report, it's not the mod, but the VirusTotal's scanner are downloading the zip file from their own servers. The same page will also record the behavior of the scanner to decompress the DSN zip with 7z, which proves that not all the behaviors recorded are from the scanning target.
If you still have doubts about the safety of the mod, you can review its code yourself or ask experienced people. All of our releases are compiled from the following repositories, using Visual Studio Community 2022, to the x64 Release target: https://github.com/YihaoPeng/DragonbornSpeaksNaturally
I'm planning on doing a playthrough when I get ahold of the anniversary edition in the near future and I would love to give this another try. Just hoping I don't have to stay still and quiet while exploring or battling.
Thanks!
Any ideas on how to fix this?
When it works, it's awesome.
crash log
AE's dll does not work with SE 1.5.97 and will crash the game. Different selections in the installation dialog will cause different dlls to be released.
Note: We have released a version for AE 1.6.1170, which can be used if you need. (As before, because the virus scan issue is pending administrator review.)
damn stromcloaks , skyrim was safe before they make mess, mods was stable and good,
well , I just starting new playthrough, don't wanna break my game yet with this update, let's see ...