0 of 0

File information

Last updated

Original upload

Created by

WallSoGB

Uploaded by

WallSoGB

Virus scan

Safe to use

About this mod

Removes cell loading stutter caused by unoptimized plugins.

Requirements
Permissions and credits
Changelogs
 


What?
I know right? What?

Today we will learn about how plugins work, how Bethesda uses them, and how game is loading cells.
As you probably noticed, we have 2 plugin variants here in New Vegas land - Master and uh, non-Master (ESM + ESP, fun fact: before Skyrim, extensions don't do anything).
What's the difference?

ESPs are an unoptimized, essentially debug, variant of plugins, meant purely for development purposes. Which is why (vanilla) GECK creates them, and why the game doesn't ship with them. What makes them so bad? References in ESPs are persistent, always loaded into memory, inefficient. In addition, game is deliberately coded to skip various optimizations for ESPs, or to ignore them completely (Skin textures say hello). They can even break the game completely with some form types (example, navmeshes).

Okay, okay we get it, all our mods are badly made then and all, get to the point. So, what, ESMs are optimized in comparison?

Yes and no. They are the proper game data format in comparison, waste less memory, work as expected etc. But... by default they are not optimized-optimized.

Wall I already downloaded the mod please just say if it will break my modlist or not

You see, Bethesda has a special optimization pass for plugins that is applied when creating ESMs. You probably didn't know about this, because it's locked behind Version Control mode in GECK. A mode that's so obscure, and so Bethesda-specific, that setting it up for an average person is just a nightmare - no wonder only 2 FNV mods in existence use it.
I beg you, get to the point

What does that optimization do? It creates cell offset data for the plugin. What is that, you may ask. You see, exterior cells (and most things inside them) are not loaded on game launch like most other forms. No, no - they load in game, while you move around. This obviously means that game has to read plugins mid-gameplay.

And here's the issue: game has no clue where cells are in the file, or even if the cell it wants to load, is in the file.
This means it has to just scan the whole file, looking for a call that may as well never exist. The larger the file, the longer it takes. The more files you have, you guessed it - the longer it takes.

Cell offsets resolve this issue - plugin stores arrays of offsets that point to where each cell is in said file, so game can immediately know if a plugin contains a cell, and where to go in order to load it.


How bad is it? Without offsets, I've seen seek times as long as 20 milliseconds, *per* file (Mainly FalloutNV.esm itself)! The cell loading code works somewhat like this:
Loop through all plugins that have the worldspace form that contains the cell we want to load, searching for the cell in every one of them.
If a plugin has offset data, the lookup is pretty much immediate. No offsets? Let's go to the hard drive mines.
Please just finish it

Wait, you said FalloutNV.esm? Did Obsidian not optimize their own plugin? They did optimize them, don't worry - vanilla plugins have offsets.
Unfortunately, using ESM Fixes or TTW... removes them. xEdit also doesn't support offset creation, and will remove them on save. Yikes!


And yes, some of you may ask - isn't cell loading multithreaded? I heard the background thread handles it. And you'd be right!
One issue: 1 Thread, 5 Cells. Meaning it's one giant queue, that in no way fits within a single frame budget, so it will ultimately stall the main thread and stutter.

Now, here's a fun thing - interior cells have offsets created in-game when loading the form (interior cell forms, in comparison, are loaded at game launch). Caveat? ESM only! This isn't as bad, since you load only 1 interior at a time (and behind a loading screen), but still suboptimal.

What?
Huh?

Oh yea, that wall of text I just made? You can forget about it.
This mod removes all these issues and limitations - all plugins get their offsets generated on game launch. Both ESMs and ESPs, because I'm nice.
You are welcome, dear mod user that will leave a "doesn't work" comment in about 21.37 minutes.

Oh, also Obsidian made all I/O serialized (single threaded). Weird code that one is, the only one signed with an OEI namespace. Idk why they did it, all other (Bethesda) games don't have this.
I undid those changes as they were hurting loading times, and because this mod is multithreaded up to 32 threads, so having it all be piped onto 1 thread is a no-no + it was sitting on my drive for like a year now, having nowhere to go.


What to expect:
- Less stutter when loading cells
- Shader compilation offset generation on game launch. Don't worry - data is cached and only re-generated whenever a plugin gets modified.
- Slightly more memory used (no free lunch! All that data must exist somewhere)
- Freshly baked offset files in your MO2's Overwrite folder


What to NOT expect:
- Complete loading stutter removal (Yet. I still have few tricks)
- Game becoming good
- Me writing shorter descriptions

Is it compatible with...
Compatible with everything. This is an automated process, no user interaction required (besides installing it).

One issue I found while making this: 
MoreMojave has absolutely broken worldspace bounds caused by a rogue cell 3B09 (it's empty, looks like a misclick in GECK).
GECK would throw an error when generating cell offsets for it - I just skip mods like this, meaning no offsets will be generated.
No clue if there are any other plugins with broken bounds like this one, I sure hope not.

Fixed by the author.

I make mods, anything should I know?
- Make sure you didn't set world bounds to insane ranges like I mentioned above.
- You can include cell offset files with your mod if you want, player won't have to generate them on launch. Nothing bad will happen if you won't though.

Source Code
GitHub

Credits
Stentorious for the sick loading bar XML
xxHash