MuKen's Active Gesture Extension for VR

This is a combination mod and application with one goal: keep you from having to pause the game and go through menus whenever possible.  While it started out as a mage-oriented tool for spell casting by drawing glyphs, it has since expanded to a general VR-ization of many of the things you normally need to access menus to do in Skyrim.  Note that this is for immersion, not to make you more efficient; in terms of combat efficiency menus ARE the fastest thing since time is paused while you are in them.  The game will not pause while you are drawing glyphs or using your VR holsters, and your effectiveness in combat is based on how quickly and accurately you can gesture!  Choose your difficulty accordingly :)

Acknowledgements

Rallyeator and CylonSurfer and Billy Pilgrim who designed the glyph graphics and textures and model for the backpack and are helping improve the overall aesthetic of the mod!

Shizof, who has been a great help with SKSE integration!

Ben Otter, whose amazing OVRLay library made the overlay integration possible!

Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock, whose work on the $P recognizer powers the gesture recognition!

Vatavu, R.-D., Anthony, L. and Wobbrock, J.O. (2012). Gestures as point clouds: A $P recognizer for user interface prototypes. Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI '12). Santa Monica, California (October 22-26, 2012). New York: ACM Press, pp. 273-280.

AliWijaya, Nociconist, Template, HideMaru, MadeXMade, Eucalyp, Josefine Svegborn, Kiran Shastry, and GreenHill from The Noun Project, who provided icons for the backpack UI.

Version 2.8 changes:

• fixed bug where lockpicking counted as crime in some inappropriate situations
• immersive looting now has correct audio and counts as crime when looting stolen goods
• fixed bug where backpack carrying item ui sometimes gets stuck in hand
• extracted alternative backpack textures into separate fomod to reduce download size
• added additional backpack colors
  

Installation

This mod requires SKSE for VR!  http://skse.silverlock.org/

• Install the MageVR mod like any other Skyrim mod.
• If you want to change buttons or backpack UI colors, edit the MageVR.ini file in Data\SKSE\plugins\MageVR\MageVR.ini
• Important for left handed users: If you are using left-handed mode in skyrim, there is a toggle in the MageVR.ini file you need to set
• Start up Steam VR <-- This must be done before starting SkyrimVR, do not rely on the Skyrim launch to start SteamVR
• Start SkyrimVR through SKSE
  

• (Optional) I personally find it better to set the fVRShieldBlockEnterAngle to 10 in the SkyrimVR ini settings to keep your backpack from blocking randomly when held (it is a functional shield).
• (Optional) There is a MageVR application in Data\SKSE\plugins\MageVR, this is normally autostarted by Skyrim.  You may instead manually start it before starting Skyrim.  Doing so affects the GPU priorities: the normal automatic execution causes MageVR to be strictly outprioritized by Skyrim.  If you are straining your GPU, then MageVR will lag graphically in order to devote all available GPU time to Skyrim (non-graphical components should still function at full speed).  If you would rather MageVR try to keep up even during strained times, starting it manually will cause it to attempt to do so.  It is fairly light in resource requirement so this will not have very noticeable impact on Skyrim.  If you do execute the game this way, make sure that the MageVR app has the same level as SkyrimVR (i.e. if you normally run Skyrim as administrator, run MageVR as administrator too)
  
  

Big thanks to Youtuber Cangar who was nice enough to provide this instructional video as well:


Note: All following instructions assume you are using the default buttons.  You can remap the buttons in MageVR.ini, and adjust the instructions accordingly.

Usage: Immersive backpack inventory and looting

MageVR provides a special "backpack" item you can equip in your offhand.  You will find an option in your SkyUI MCM configuration menu to spawn this item freely (it has no value and weighs nothing).  This item can be equipped normally, or quickly using the MageVR holster slots (or any other similar equipment functions of other mods).  When it equips, an overlay will be displayed allowing you to browse your inventory.  Different inventory filtering options can be selected on the bottom by clicking on them with grip.

Note:  If you find the angle of the overlay to be off from the backpack model make sure that you have not altered shield angles in your ini settings (fLeftHandShieldRotationRoll=-60.0000 and fRightHandShieldRotationRoll=60.0000), and that you have the 'realistic' shield option set in the in-game VR menu.

To drop any item, drag it out of the backpack and release it away from your body in front of you.  To use an item, release it at face level (this will allow you to drink potions, cast scrolls, and apply poisons).  To equip an item, release it on your own body.  Weapons that are equippable in either left or right hand can be equipped on either left or right side to go into the corresponding hand.  Note that if you equip it into your offhand, this will unequip the backpack and close the inventory.

Using "equip" on a potion or poison will slot it into the MageVR grip gesture slot for that type and hand (for more information see below).  Repeating the same equip action will instead slot it into the thumbstick gesture slot for that type.

To immersively loot an unlocked container or corpse, simply activate it while your backpack is in hand.  Instead of the normal paused looting window, you will get a floating display in realtime of the loot.  Items can be dragged back and forth between your backpack and the loot window, and can be directly used/equipped/dropped with the same gestures as from the backpack.

Both your backpack and the loot window will show different colors for value per weight ratio of all items to help you quickly identify what you should loot.  These colors and the thresholds of the ratios can be reconfigured in the ini file.  By default they are:

Grey: <5 gold/weight
White: 5-10 gold/weight
Green: 10-20 gold/weight
Blue: 20-50 gold/weight
Purple: 50-100 gold/weight
Gold: >100 gold/weight




Usage: Immersive lockpicking

To immersively pick a lock, activate it while your MageVR backpack is equipped.  You will see a lock and pick appear in your hands.  Locks have from 3-7 tumblers based on lock difficulty, your goal is to insert the pick and push them all up in the correct order (order is unique per lock and rerandomizes if you reload your game).

You will know which tumbler your pick is underneath by counting “click” sounds as you insert the pick.  You should test each tumbler by gently pushing up on it with the pick, a squeaking sound and hard haptic feedback indicates you are pushing the wrong tumbler.  The correct tumbler will have gentle haptics as you push on it.  Once you push it far enough, you’ll hear a different progress sound, indicating you should find the next.  Pushing the wrong tumbler damages your lockpick, though you can mitigate it somewhat by being careful not to push too far.  If your lockpick breaks, all the tumblers will reset, but the order will remain the same so you can repush the ones you’ve figured out so far.

Be aware that unlike the base lockpicking, time is not paused during immersive lockpicking, and guards and other people will react to your criminal activities!

Usage: Glyph casting

Hold the left grip to enter "draw mode".  You can now draw strokes by pressing your left and right triggers.  When you release the left grip, both hands will equip spells according to the drawn glyph.  Alternatively, hold the right grip to enter "quick cast draw mode".  Now whatever glyphs you draw will be instacast in the direction you are looking and your equips will not change.

The spells you are drawing can be freely slotted.  To do so, first equip the spells you want to slot.  Then draw a glyph while holding right grip (i.e. quick cast mode), then while still holding right grip, briefly click your left grip.  This will slot the spells in both hands to the slot of the glyph you just drew.  To confirm, when you release the right grip for quick cast, those spells will cast.

The list of available glyphs comes from a set of 7 "core glyphs".  Each of these core glyphs can be drawn, and then from 0 to 4 additional "modifier strokes" can be drawn on top of them to get different glyphs.  This gives you a total of 7 * 2^4 = 112 glyph slots.  The 7 core glyphs and 4 modifier glyphs are as shown below:





Usage: Handpose magic

You can also equip and cast spells by forming poses with your controllers.  If using this system, then having Immersive Sounds installed will allow it to play sound effects on successful handposes, which will make it much easier and more immersive to use.  You can either use this together with the glyph system, or edit the buttons in MageVR.ini to block one or both systems and just use what you want.

Handpose slots are similar to glyph custom slots, except they function based on handpose sequences.  To perform a handpose sequence, hold one of the handpose activation buttons, then do a sequence of handposes while holding your hands in your center line in front of your chest.  Slots sequences include the spell set up of both hands (they are not handled individually as with the glyphs, since handposes use both hands).

To save a slot, hold the right grip, do your handpose sequence, then click the left grip (while still holding the right grip), just as with the custom glyphs.  This will save both hands' equipped spells.  To reequip them later, hold the left grip, do your handpose sequence, then release it.  To quick cast what you have in both slots, hold right grip, do your handpose sequence, then release right grip.

There are four distinct handposes:

• '=' sign 'SAH': left hand pointing to the right and right hand pointing to the left
• 'T' sign 'TZE': one hand pointing up, the other hand above it pointing across
• '┴' sign 'NIH;: one hand pointing up, the other hand below it pointing across
• 'l' sign 'JUL': both hands pointing up, one above the other
  

Your handpose sequence can use 1-3 poses, for a total of 4 + 16 + 64 = 84 slots.  If you are unfamiliar with pointing direction, try equipping two daggers at first, to see which way SkyrimVR considers your hands to be pointing; it will always match the direction weapons face in.  The 'up' direction is relative to the ground, not your headset, so if you are looking down at your hands, make sure you don't tilt your hands forward.

If you need to repeat a handpose in sequence, swap your hands to register a second handpose.  I.e. if you are making the '=' sign with left hand on top, do it again with right hand on top to get a second '='.

If you are attempting to transition between the '=' sign and the 'l' sign without accidentally doing one of the other signs in between, you can separate your hands from center before turning them.  You can also turn them to point forwards since this pose is not used.

Usage: Shoulder holsters/quivers/quickslots

There are three holsters over each shoulder (for a total of 6).  Think of holsters as real life sheathes, they can hold exactly one weapon at a time, and you can take that weapon out of it or put it back in.  To use one, hold your hand over that shoulder and press either trigger or grip or click the thumbstick to use the three different slots.  Whatever is equipped in that hand will go into the holster (and no longer be equipped), and whatever is in the holster will go into the hand.  Holsters can hold weapons, shields, staffs, and torches.

For example, if you have a torch in your left hand and a sword over your left shoulder trigger slot, clicking trigger while holding your hand over your left shoulder will equip the sword, and now that holster will have a torch in it.  If your hand was originally empty, that holster will now be empty and the sword will have equipped into your left hand.  Although the option to quickswap your held weapon with the one in holster is provided, most people find it less confusing to mentally associate each holster with one weapon.  Only draw from it with an empty hand, and put the weapon back in its holster (emptying your hand again) before drawing a different weapon from another holster.  Treat them like real life holsters.

Holsters consider spells to be empty hands, when you pull a weapon out it will go "on top" of your currently equipped spell, and that spell will return when you re-holster the weapon.

In the SkyUI MCM configuration menu, you will find an option to convert your holsters into quivers for any arrows you have.

The same menu allows you to convert a holster into a quickslot for the weapon that is currently in it.  Quickslots are fixed, they will not change contents when used.  If you wish to later reassign it, convert it back into a holster, put in a different weapon, then convert it into a quickslot for the new weapon.

Realistic/Hardcore archery

There are two optional modes to make archery more immersive.  This can also be configured via SkyUI MCM.

Realistic archery causes your arrow to unequip every time you fire your bow.  This is meant to be used in conjunction with the shoulder quiver so that you need to immersively draw a fresh arrow for every shot.

Hardcore archery uses stamina penalties to force you to wait for a certain period after drawing an arrow before firing it.  The amount of delay is equal to the delay dictated by the firing speed of the weapon in vanilla Skyrim (Quick Shot perk is respected).  Note that since the animation for nocking an arrow is replaced by your vr action, it is still possible to fire faster than the vanilla Skyrim speed.  A bow tension sound effect will start as soon as you draw an arrow and stop to let you know when it is ok to fire.

Firing faster than this limitation gives you a stamina penalty.  The stamina penalty increases based on how much faster you are firing, and also how much stamina you are already missing (so as you continue to fast fire, your stamina depletes faster and faster).  If your stamina gets too low, you will no longer be able to draw fresh arrows.

Usage: Potion drinking

Clicking trigger over your mouth will quaff a potion.

Use your left hand to drink a health potion (or eat food).  Use your right hand to drink a magicka or stamina potion; it will choose based on which meter is missing more.  If you do this while in combat, it will pick the one that is closest to the amount you are missing.  If you do this out of combat, it will pick the smallest one you have.

Alternatively, you have two additional slots per hand by using grip or thumbstick buttons.  These can also be configured via SkyUI MCM to hold any of the potions you have in your inventory.

Usage: Poisons, scrolls, and recharge

You also have four slots divided between your two wrists, accessible by clicking with grip or thumbstick with your other hand.  For example, move your right hand to your left wrist and click grip to activate a slot.  Similarly to the potions, these can also be configured via SkyUI MCM.

Note: Skyrim always pops a confirmation window when using a poison.  If you leave the Skyrim VR window on top, then MageVR will automatically accept this confirmation for you by sending a keystroke.

This can also be used to slot two recharge functions (requires  the GIST mod to be installed and loaded before MageVR).  The "smallest recharge" function will use the smallest completely filled soul gem you have on the hand whose wrist you slotted it.  If that hand is not holding a rechargable weapon, it will attempt to charge the weapon in the other hand.  The "targeted recharge" function will use the largest soul that doesn't waste any charge.

Usage: Shouts

You also have six slots between your two hands.  To use these, thrust both hands down and back and your head forward, then click one of the three buttons on either hand.  Doing this while a shout is equipped will unequip that shout and put it into the slot.  Doing this while no shout is equipped will execute the shout you slotted.  This can also be configured via SkyUI MCM to use different numbers of shout words.

Note:  This functions by sending a keystroke to SkyrimVR from MageVR application; it requires the SkyrimVR window to have keyboard focus.

Usage: Powers

Powers are activated with the same six buttons you mapped for shouts, but with a different gesture (and stored in another 6 slots as well).  The power activation motion is to cross your arms and hold one of the six shout buttons (hold this for a brief period, 1/10 a second is sufficient), then continuing to hold the button, uncross your arms, bringing your left hand to where your right hand was and your right hand to where your left hand was.

Just as with the shouts, doing this while a power is equipped will unequip it and assign it to the slot you have activated.  Doing it while no power is equipped will activate the power you have slotted.

As with shouts, this requires the SkyrimVR window to have keyboard focus.