This project is licensed under the GNU Affero General Public License v3 (AGPL-3.0-or-later). See the LICENSE.txt file (https://github.com/jwebmeister/tacspeak/blob/main/LICENSE.txt) or https://www.gnu.org/licenses/agpl-3.0.html for details.
File credits
- Based upon and may include code from "Dragonfly" [dictation-toolbox/dragonfly](https://github.com/dictation-toolbox/dragonfly), under the LGPL-3.0 license. - Based upon and may include code from "Kaldi Active Grammar" [daanzu/kaldi-active-grammar](https://github.com/daanzu/kaldi-active-grammar), under the AGPL-3.0 license.
you can also review the same file to understand and/or modify the commands; it should be intuitive or map to the in-game command menu, but your mileage may vary.
(Note: you will need to restart Tacspeak for changes to take effect.)
Commands list (imperfect, outdated, not maintained, but maybe useful)
Important advisory
Please use caution and your own discretion when installing or using any third-party files, specifically *.py files. Don't install or use files from untrustworthy sources.
Tacspeak automatically loads (and executes) ./tacspeak/user_settings.py and all modules ./tacspeak/grammar/_*.py, regardless of what code it contains.
Open ./tacspeak/user_settings.py in a text editor, change the settings, then save and overwrite the file. There are comments in the file explaining most of the important settings.
None = overrides listen_key_toggle, and sets it into always listening mode; uses Voice Activity Detector (VAD) to detect end of speech and recognise commands.
listen_key_toggle=-1
Recommended is 0 or -1.
0 for toggle mode off, listen only while key is pressed; must release key for the command to be recognised.
1 for toggle mode on, key press toggles listening on/off; must toggle off for the command to be recognised.
2 for global toggle mode on, key press toggles listening on/off, but it uses Voice Activity Detector (VAD) to detect end of speech and recognise commands so you don't have to toggle off to recognise commands.
-1 for toggle mode off + priority, listen only while key is pressed, except always listen for priority grammar ("freeze!") even when key is not pressed.
vad_padding_end_ms=250
change this if you use VAD and find it's too quick or slow to identify the figure out you've stopped speaking and that it should try to recognise the command.
audio_input_device=None
should use default microphone (as set within Windows Sound Settings), but should be able to change the index (number) to select a different input device.
USE_NOISE_SINK=True
load NoiseSink rule(s), if it's setup in the grammar module - it should partially capture other noises and words outside of other rules, and do nothing. Set to False if you're having issues with recognition accuracy.
retain_dir= ./retain/
use this setting to retain recordings of recognised commands - set to a writeable directory to retain recognition metadata and/or audio data. Disabled by default.
retain_audio= True
use this setting to retain recordings of recognised commands - set to True to retain speech data wave files in the retain_dir. Disabled by default.
retain_metadata= True
use this setting to retain recordings of recognised commands - set to True to retain metadata of recognitions in a .tsv file in retain_dir. Disabled by default.
retain_approval_func= my_retain_func
use this setting to retain recordings of recognised commands - set to a function returning True or False based on AudioStoreEntry contents. Disabled by default.
Grammar modules
It is likely you will want to modify or customise some of the existing Tacspeak grammar modules (if not also add your own!), which you can do by editing the ./tacspeak/grammar/_*.py file corresponding to the application you're interested in.
As an example, in the Ready or Not module you can change ingame_key_bindings to align the Tacspeak module with your in-game keybindings.
You could also change the words and/or sentences used for recognising speech commands, for example, adding "smoke it out" as an alternative to "breach and clear".
Additional notes:
Please see the existing examples of Tacspeak grammar modules.
Please see the Dragonfly docs for information on building grammars, rules, and actions (i.e. voice commands).
Note: Tacspeak uses a modified version of Dragonfly located at jwebmeister/dragonfly. Review the source and/or commits of the fork to understand its differences to the original project and the corresponding docs.
Have the Tacspeak application and model files been extracted into the correct locations? Check the following files exists:
./tacspeak.exe
./tacspeak/user_settings.py
./tacspeak/grammar/_readyornot.py
./kaldi_model/Dictation.fst - if not you need to download and extract the pre-trained model
Have you tried "Run as administrator" on tacspeak.exe?
Is the correct microphone set as the default in Windows Sound Settings?
Are you pressing the listen_key (by default it is mouse thumb button), and does it show "Hot mic" in the console?
Are you running ReadyOrNot and have the window focused (i.e. you're not alt-tabbed to another window)?
Are you pressing the listen_key (default is mouse thumb button), speaking, then releasing after you finish speaking?
Check the key bindings in ./tacspeak/grammar/_readyornot.py. It's set for default game keybindings.
Check the ".tacspeak.log" file for any useful error messages to narrow it down.
Try reinstalling (extracting from .zips) everything, including the model, don't change anything in ./tacspeak/user_settings.py or ./tacspeak/grammar/_readyornot.py keep it all default, try running tacspeak.exe.
For Tacspeak version ≥0.1.1, run ./tacspeak.exe --print_mic_list in Powershell or command prompt.
This will list all of the audio devices found on your system, and can be useful for figuring out the correct index number for the audio_input_device setting in ./tacspeak/user_settings.py.
A far easier option to try first is to set the correct default recording device in Windows Sound Settings.
The underlying model that Tacspeak currently uses is based on "16-bit Signed Integer PCM 1-channel 16kHz" audio. Tacspeak tries to convert the incoming audio from your device to this format, but if it's too much for a single CPU core to convert in real-time it may fall over.
I've had no issues using Tacspeak with a 48kHz, 16-bit, 2-channel microphone array and also using a Rode AI-1 and Podmic at 48kHz, 24-bit, 1-channel.
If, for example, your device is recording at 144kHz, or something a single core on your CPU can't handle, it will likely display errors in the console.
If you no longer hear audio from your output device (headphones), or no audio is coming through from your input device (microphone), you might have to disable "Exclusive Mode" for your audio device in Windows Sound Settings. Follow these steps to disable Exclusive Mode:
Right-click the Speaker icon on the Windows toolbar, and select Open Sound settings.
Click Device properties located underneath Choose your output device, then click Additional device properties located underneath Related Settings.
In the Line Properties window, click the Advanced tab, then uncheck Allow applications to take exclusive control of this device.
Click Apply, then click OK.
All commands are being queued? AZERTY keyboard? You might need to change your in-game keybinding for "Hold command" to something other than Shift, and also update the key bindings in ./tacspeak/grammar/_readyornot.py.
In your Windows privacy and security settings, can apps (including Tacspeak) access your microphone? Is your microphone working in other apps?
Try setting USE_NOISE_SINK (in ./tacspeak/user_settings.py) to True or False if you're getting too many false positive or false negative speech recognitions respectively.
If you want to retain recordings of the speech recognitions from Tacspeak, set retain_dir, retain_audio and retain_metadata (in ./tacspeak/user_settings.py) appropriately.
Motivation
I built Tacspeak because I was fed-up with how poorly accurate the Windows Speech Recognition engine was with my voice, even after training. No other alternatives I tested (there were many) fit exactly what I wanted from speech recognition while gaming.
What I learned from my research and testing:
most state-of-the-art Automatic Speech Recognition (ASR) systems are not fit for the purpose of speech command recognition while gaming. They:
take too long, e.g. 1-3 seconds.
take too much memory, e.g. 2-4 GB of VRAM (textures pop-in in-game).
take too much CPU/GPU processing power.
are designed for wider applications beyond speech command recognition, e.g. free-form dictation.
there are decades-old ASR's that are fit-for-purpose, but their tool and build chains were too unwieldy.
on paper, the Windows Speech Recognition engine should be perfect for my use-case, it just hates me (and everyone else).
I needed a customisable speech recognition framework to fit my specific use-case.
Tacspeak isn't perfect, but it is a very strong option, precisely because it can be so highly customised to your specific commands, for your specific application.
Contributing
Issues, suggestions, and feature requests are welcome on the github repo jwebmeister/tacspeak.
Pull requests are considered, but be warned the project structure is in flux and there may be breaking changes to come.
We'd also like some (TBD) quality testing be done on grammar modules before they're brought into the project. If you can help define what we mean by "some (TBD) quality testing"... well, trailblazers are welcome!
Tacspeak uses a modified version of Dragonfly located at jwebmeister/dragonfly. This is where the heart of the beast (bugs) lives... please help slay it!
Also, be warned the project structure is in flux and there may be breaking changes there too.
