I have decided against uploading fixes for specific mods, at least for now. It would be impossible to make sure that the Pyffi optimized meshes all work correctly, especially for larger mods.
However, I intend to keep updating the lists of affected and unaffected mods, at least for the time being. I also plan to write a guide, so people can detect the issue themselves and attempt to fix it in mods they are using. Of course, it would be best if mod authors would provide fixed versions themselves, but of course, many are no longer active.
Meanwhile, I tried fixing the issue with NifSkope, but triangulating all NiTriStrips or stitching strips both led to the game just hanging on loading a cell with the mesh (in this case, Cheydinhal), and crashing after a while. So the only way that works appears to be Pyffi. Edit: Triangulate apparently works, and it may be safer than Pyffi in some cases, because only the problematic NiTriStrips are edited and the rest of the file should remain unchanged. NifSkope 2.0 can batch triangulate all strips, while in NifSkope 1.1.3, each NiTriStrips has to be done manually.
Edit (again): I didn't realize that Sniff could not only find meshes with this issue, but it can also convert NiTriStrips to NiTriShapes ("Triangulate" in NifSkope), which is the easiest and fastest method, and possibly also the safest. But I have to test it a bit more to be more certain. Be aware that Sniff will copy the changed files into the specified Output directory, you have to move them back to their original folder path manually!
However, NEVER simply use Pyffi on ALL meshes of a mod or your entire "meshes" folder, this will almost certainly break something! Find problematic meshes with Sniff, and make backups before using Pyffi on them! Best avoid character models, creatures, possibly armor and clothing as well, or you might break their animations, which can make Oblivion crash! Multiple stips in NiTriStrips can decrease performance, but that is still better than a game that crashes! If possible, check if the meshes still work immediately after using Pyffi, and only do one, or a few, at a time! Otherwise it could become very difficult to find out which model exactly causes a problem!
That's true, but QTP3 hasn't been updated as well. I guess it's possible that there are newer changes in the UOP that conflict with the QTP3 patch, but it's only meshes, anyway, so nothing that should really break anything. Anyway, I've been told that they are going to update it, but they want to check all meshes first.
I was asking this because I'm actually following Bevilex's modlist. I didn't want to use OUT as I was unfamiliar with it and wanted to use a mod I was accustomed to, which is QTP. UOP got and update but both QTP and the QTP UOP Patch didn't, so I was just curious. Thanks anyway
I realize I'm 3 years late on this one but I figured I'd mention it anyway.
QTP3 itself is unlikely to get updated again as the author passed away some years ago.
The QTP3 patch for UOP is something that still needs to be looked into and I'll poke the guys to see what's what here and what, if anything, we may still need to do.
Thank you for making this. I posted about this problem on the UOP page months ago and was rudely told by Arthmoor that there was no way UOP could be the problem, even though I showed clear evidence. Its very unfortunate how offended mod authors can be when a user points out a potential legitimate problem that has affected many. At least the UOP team could have investigated this problem for themselves after receiving multiple reports, I know I was not the only one that brought this up. Big sigh
I know this comment is like 2 years old lol but like this is so true.
As an example, I pointed out on the Unofficial SSE patch that it literally broke dialogue for 2 Master Trainers in the Thieves Guild, even left a screenshot of xEdit showing how I was able to fix the problem by reverting their unnecessary changes and got told "Bold of you to assume this is caused by the Unofficial patch"
Which was such an audacious, retarded response especially considering I'd even left a screenshot for those morons as evidence as to what was causing the issue and how to fix it.
The unbelievable arrogance of the moderators for USSEP never ceases to amaze me.
Arthmoor himself, surprisingly, wasn't actually that rude about it though. He commented on my 2 uploaded fix mods for the broken dialogue acknowledging that, it is indeed the USSEP causing the problem and that it'd be fixed in the next hotfix.
So in regards to the Unofficial Oblivion Patch incorporating this, how should I preoceed to install Bomrets Shivering Isles Texture Pack with Meshes? Should it go before UOP, or should I just stuck with SIUT? (Shivering Isles Upscaled Textures)
Well, I don't know if there are other meshes that conflict with the unofficial Patches, so I cannot say for sure. But as I wrote in the description there are only two meshes, of which one is not even used in the game, with this issue. So you could either fix it yourself with Sniff, or you could use the UOP version instead. Which texture mod you should use is up to you.
So, I can just use Bomret's Shivering Isles no problem? "or you could use the UOP version instead" What do you mean? UOP doesn't have a compatibility patch for Bomret's, or were you inferring something else?
The thing is, I don't know if Bomret's Pack makes any additional changes to the meshes, so I'm unsure if the meshes from the Unofficial SI Patch can be used instead without any issues (like misaligned textures or something, but as I said, I don't know if there are such differences). However, they do recommend using this mod, which is now included in the Unofficial Patches, so it might be ok. But if you want to be on the safe side, use the meshes from Bomret's, except the one with this problem that is used by the game: SEWallPost01.nif. Use the version from the Unofficial Patch (or this mod, if you like) instead.
Hi, the main file of this mod (which I just moved to "Old Files") is no longer needed, as the UOP has been fixed and no longer has this issue. However, the QTP3 patch hasn't been updated, so if you are using it, you should install the patch from this mod after it and overwrite the files.
Do you believe that (for an equal number of triangles), the position of a vertex would have an influence on performance? On those pieces, the angles are much sharper on the right. (Left is from Oblivion, Right is from a mod)
As far as I'm aware, the position of individual vertices and polygon angles have no impact on performance. Unless they have the "multiple strips" issue that this mod is trying to fix, I don't think these simple objects should cause any problems.
Which mod are you trying to troubleshoot and what are your problems with it?
I'm playing Morroblivion. I send a couple of bugs regarding quests, and I installed Blender 2.49b, looking at meshes and collision. In Arrille's Tradehouse, on the ground floor, I have 60ish FPS. Go up a level, and turn towards the table and chairs, and the FPS drops around 30. The models there have a lot of triangles. The collision are also "high poly". I understand why that would be a problem. I'm also intrigued by the outside door. When I walk around the village, the FPS are good, but when I stand facing the door, nothing else on the screen, I also drop under 30.
I haven't played Morroblivion at all, so I can't really comment on this problem, I'm afraid. However, regarding the door issue, even if you can't see anything else on screen, the game is still rendering everything that is behind that door, including NPCs on the other side of the building. That may have something to do with the FPS drop. You can also try disabling objects with the console to check if it makes a difference whether they are there or not.
Thank you for this. A couple questions/comments, and I apologize if you've already answered this somewhere.
1) As a matter of course I run every mod including vanilla through PyFFI (specifically, the built-in oblivion_optimize toaster for PyFFI 2.2.2 on Python 3.2.5 x64, as I cannot find an installer for the latest PyFFI, 2.2.4, nor can I find any version compatible with Python 3.3 or higher). Based on your knowledge, do you see any reason not to do this? I haven't encountered any issues in-game doing so.
2) I always use the default settings/INIs for the above. Any reason to change?
3) Do you think there would be any benefit to running meshes through PyFFI multiple times? I don't know if the scripts are recursive (i.e. continue cycling until no further changes are found), or not.
4) Since the Unofficial Patches appear to be final, I thought about uploading an all-in-one package for Oblivion, SI, the DLCs, and the Shambles crash fix that's not included, with PyFFI'd meshes and Ordenador'd textures to deal with the issue you've described here, and others. The Patch permissions would allow this. However, I don't want to "steal your thunder", and so suggest that you consider doing this instead. It would help newbies to have one place to go instead of multiple! (Side note: I also thought about throwing the new Unofficial assets into a BSA but there are only 15MB or so that could safely be added to one. So it's probably not worthwhile.)
1) If you have never run into problems, good for you. It is possible, though, that meshes can break and cause crashes afterwards, and I have encountered this myself.
2) Not that I'm aware.
3) Probably not. I think running Pyffi once is enough. I've often read that running it multiple times on the same mesh could be harmful, but I don't know if that's true.
4) I was contacted by the UOP authors about this, and it's possible that they'll release an update, but I don't know when and what exactly that will include. If you want to upload a package yourself, go ahead. I no longer have the time to mod or play Oblivion, so I won't do it myself.
Thanks for the reply. On 1: No, I've never run into any problems batch-running PyFFI when utilizing the latest oblivion_optimize batch files, which exclude all known problematic files (Night Mother statue, hair, etc.). There's always a first though, and I do keep backups just to make certain (which, to make fair, you make abundantly clear is a good idea - hopefully people will heed the warning).
And yes, it looks like the unofficial patches have been updated to incorporate your fixes! That is unexpected and appreciated.
Ah, I wondered who was "in charge" nowadays. I'll message them as well - they may want to add the Shambles knockdown CTD fix to USIP while they're at it. Thanks again for your work on this; most people would have been none the wiser.
Just to add my two septims, I'd like to share some findings from my recent PyFFI tests, where I optimised all the vanilla oblivion meshes with different combinations of PyFFI and Python.
I've found that PyFFI 2.2.2 is the safest bet for Oblivion meshes. Newer PyFFI versions have some big changes under the hood to add support for NIF formats used by newer games, but there is a small risk for regressions. In practice, I found very little difference, but newer PyFFI versions did fail on a few meshes that are in an older NIF format. They also added an extraneous collision block for a few meshes, and when I asked the developers about it, one of them suspects that 2.2.2 is doing it right. The only thing you might need newer versions for is if you want to do mass texture path substitutions, since 2.2.2 has a bug related to that.
As for Python versions, 3.8 removes a long-deprecated time-related function, which breaks older PyFFI versions. Any Python version between 3.2.x and 3.7.x should be fine to use with PyFFI 2.2.2. Personally I settled on Python 3.3.5, since that's contemporary with PyFFI 2.2.2. I didn't find any significant difference in the output of different Python versions; it was mostly just a few triangles sometimes being saved in a different order. By the way, PyFFI 2.2.2 can work with newer Python versions than 3.2.x, it's just that the installer for the Toaster isn't set up for it. Just installing the Toaster manually works well, however.
Regarding optimising the same mesh with PyFFI multiple times, I don't think it should cause any problems. An older version of PyFFI famously had a problem with broken collision, and running an already broken mesh through newer versions probably won't fix that; for that you'd have to start over with an unmodified mesh. Using a recent PyFFI version multiple times on a mesh should be fine; if the mesh is already optimised it just shouldn't do anything. It probably also doesn't do any good though. PyFFI versions before 2.2.2 had a bug where you sometimes had to run it twice to catch all optimisation possibilites, but that should be fixed now.
Lastly, regarding the skip list, I've found that it's quite conservative. PyFFI can handle the Night Mother statue just fine now, for example, and it's also safe to optimise closed-face helmets, since they don't have an associated EGM file. Even some meshes with hair in the name are safe, as long as they are for creatures, such as boars, trolls and zombies, rather than for characters.
Interesting! I appreciate the thorough analysis here. I was aware of the deprecation of the time function in 3.8 but wasn't sure if that was the only deprecated function that caused incompatibility, nor which of the two replacement functions would be more appropriate. I also wasn't sure if any of the other used functions might have changed between 3.3.x and 3.10.x such that the code still technically ran but gave different outputs. So I started editing the Python scripts to update them but then decided against it. (Still, it's annoying to need two different Python 3.x installs, one for PyFFI and one for everything else. Ah well.)
It seems I stumbled across the best solution, at least for Oblivion, by sticking with 2.2.2 and 3.3.x without being aware of it. Your comment regarding texture path substitutions does worry me, since (besides the issue with vanilla assets that have a leading space in the name) that is one of the spells that executes in the automated batch. But I've not noticed any missing (i.e. purple) textures to date, so hopefully that means all is well.
Interesting, too, that the Night Mother statue and other meshes long on the "do not touch" list may be safe now. I may experiment with those. I also hadn't noticed that closed helmets lack EGMs but it makes sense that they wouldn't need them.
To clarify, the fix_texturepath spell used by the Toaster for optimisation should be fine in 2.2.2; it's the substitute and modify texturepath spells where you input a new path manually that seem to have problems. NifSkope and Sniff should be able to handle that fine instead though, so it's not a big issue.
Ah, the MODIFY texturepath, not the FIX texturepath. I missed that. Thanks for clarifying!
Between PyFFI, Ordenador, TES4LodGen and LandMagic, I think I'm in pretty good shape on the meshes and texture end now. I've considered running all the voice MP3s through a volume-rebalancing algorithm (MP3Gain v2, specifically), but that requires the assumption that every file is supposed to be at the same volume. Which isn't true. So there's probably no way to automate it.
I've tried to fix meshes of the mods in your list and Cobl seems to break. I CTD in player houses after using Sniff on its meshes (not all of them, just the affected ones). I don't really want to dig into this so I just left Cobl untouched for now.
In your posts, you mentioned about different tools for this kind of mesh optimization. Is it possible that using something different than Sniff could yield better results? If so, what's my best bet in case I wanted to try fixing some meshes again? Sorry but although I work in IT, I'm not knowledgeable about 3D modeling.
It's possible, yes. So far I haven't had any issues with manually triangulating NiTriStrips with NifSkope, though that can be very tedious. NifSkope 2.0 has a function to triangulate all NiTriStrips, but that has also caused issues for me. The same is true for Pyffi. As far as I'm aware there is no way to do it safely on a larger number of meshes. And in many cases there probably won't be a big performance improvement. The Cobl meshes for the most part only have a relatively low number of strips, and unless there are many of them present at the same time they probably won't cause noticeable issues.
Yeah, I don't really care about COBL. That's just the one that crashed after I stupidly optimized everything I found (Mesh Improvement Project works though). I was rather thinking about looking through my other mods, especially ones with architecture meshes. I'll start with checking my draw calls with Reshade in areas where my FPS tanks as you said and proceed from there.
122 comments
all work correctly, especially for larger mods.
However, I intend to keep updating the lists of affected and unaffected mods, at least for the time being. I also plan to write a guide, so people can detect the issue themselves and attempt to fix it in mods they are using. Of course, it would be best if mod authors would provide fixed versions themselves, but of course, many are no longer active.
Meanwhile, I tried fixing the issue with NifSkope, but
triangulating all NiTriStrips orstitching stripsbothled to the game just hanging on loading a cell with the mesh (in this case, Cheydinhal), and crashing after a while. So the only way that works appears to be Pyffi. Edit: Triangulate apparently works, and it may be safer than Pyffi in some cases, because only the problematic NiTriStrips are edited and the rest of the file should remain unchanged. NifSkope 2.0 can batch triangulate all strips, while in NifSkope 1.1.3, each NiTriStrips has to be done manually.Edit (again): I didn't realize that Sniff could not only find meshes with this issue, but it can also convert NiTriStrips to NiTriShapes ("Triangulate" in NifSkope), which is the easiest and fastest method, and possibly also the safest. But I have to test it a bit more to be more certain. Be aware that Sniff will copy the changed files into the specified Output directory, you have to move them back to their original folder path manually!
However, NEVER simply use Pyffi on ALL meshes of a mod or your entire "meshes" folder, this will almost certainly break something! Find problematic meshes with Sniff, and make backups before using Pyffi on them! Best avoid character models, creatures, possibly armor and clothing as well, or you might break their animations, which can make Oblivion crash! Multiple stips in NiTriStrips can decrease performance, but that is still better than a game that crashes! If possible, check if the meshes still work immediately after using Pyffi, and only do one, or a few, at a time! Otherwise it could become very difficult to find out which model exactly causes a problem!
QTP3 itself is unlikely to get updated again as the author passed away some years ago.
The QTP3 patch for UOP is something that still needs to be looked into and I'll poke the guys to see what's what here and what, if anything, we may still need to do.
As an example, I pointed out on the Unofficial SSE patch that it literally broke dialogue for 2 Master Trainers in the Thieves Guild, even left a screenshot of xEdit showing how I was able to fix the problem by reverting their unnecessary changes and got told "Bold of you to assume this is caused by the Unofficial patch"
Which was such an audacious, retarded response especially considering I'd even left a screenshot for those morons as evidence as to what was causing the issue and how to fix it.
The unbelievable arrogance of the moderators for USSEP never ceases to amaze me.
Arthmoor himself, surprisingly, wasn't actually that rude about it though. He commented on my 2 uploaded fix mods for the broken dialogue acknowledging that, it is indeed the USSEP causing the problem and that it'd be fixed in the next hotfix.
Forgive me, I'm just confused.
I'm trying to troubleshoot an existing mod.
Which mod are you trying to troubleshoot and what are your problems with it?
In Arrille's Tradehouse, on the ground floor, I have 60ish FPS. Go up a level, and turn towards the table and chairs, and the FPS drops around 30. The models there have a lot of triangles. The collision are also "high poly". I understand why that would be a problem.
I'm also intrigued by the outside door. When I walk around the village, the FPS are good, but when I stand facing the door, nothing else on the screen, I also drop under 30.
A Google search brought me here, so I hope you will keep the mod up, even if it's not necessary anymore.
1) As a matter of course I run every mod including vanilla through PyFFI (specifically, the built-in oblivion_optimize toaster for PyFFI 2.2.2 on Python 3.2.5 x64, as I cannot find an installer for the latest PyFFI, 2.2.4, nor can I find any version compatible with Python 3.3 or higher). Based on your knowledge, do you see any reason not to do this? I haven't encountered any issues in-game doing so.
2) I always use the default settings/INIs for the above. Any reason to change?
3) Do you think there would be any benefit to running meshes through PyFFI multiple times? I don't know if the scripts are recursive (i.e. continue cycling until no further changes are found), or not.
4) Since the Unofficial Patches appear to be final, I thought about uploading an all-in-one package for Oblivion, SI, the DLCs, and the Shambles crash fix that's not included, with PyFFI'd meshes and Ordenador'd textures to deal with the issue you've described here, and others. The Patch permissions would allow this. However, I don't want to "steal your thunder", and so suggest that you consider doing this instead. It would help newbies to have one place to go instead of multiple! (Side note: I also thought about throwing the new Unofficial assets into a BSA but there are only 15MB or so that could safely be added to one. So it's probably not worthwhile.)
Thanks.
1) If you have never run into problems, good for you. It is possible, though, that meshes can break and cause crashes afterwards, and I have encountered this myself.
2) Not that I'm aware.
3) Probably not. I think running Pyffi once is enough. I've often read that running it multiple times on the same mesh could be harmful, but I don't know if that's true.
4) I was contacted by the UOP authors about this, and it's possible that they'll release an update, but I don't know when and what exactly that will include. If you want to upload a package yourself, go ahead. I no longer have the time to mod or play Oblivion, so I won't do it myself.
And yes, it looks like the unofficial patches have been updated to incorporate your fixes! That is unexpected and appreciated.
It looks like the QTP3 patch has not been updated, yet. I'll ask SigurdStormhand about that, if they just missed it or intend to update that as well.
Just to add my two septims, I'd like to share some findings from my recent PyFFI tests, where I optimised all the vanilla oblivion meshes with different combinations of PyFFI and Python.
I've found that PyFFI 2.2.2 is the safest bet for Oblivion meshes. Newer PyFFI versions have some big changes under the hood to add support for NIF formats used by newer games, but there is a small risk for regressions. In practice, I found very little difference, but newer PyFFI versions did fail on a few meshes that are in an older NIF format. They also added an extraneous collision block for a few meshes, and when I asked the developers about it, one of them suspects that 2.2.2 is doing it right. The only thing you might need newer versions for is if you want to do mass texture path substitutions, since 2.2.2 has a bug related to that.
As for Python versions, 3.8 removes a long-deprecated time-related function, which breaks older PyFFI versions. Any Python version between 3.2.x and 3.7.x should be fine to use with PyFFI 2.2.2. Personally I settled on Python 3.3.5, since that's contemporary with PyFFI 2.2.2. I didn't find any significant difference in the output of different Python versions; it was mostly just a few triangles sometimes being saved in a different order. By the way, PyFFI 2.2.2 can work with newer Python versions than 3.2.x, it's just that the installer for the Toaster isn't set up for it. Just installing the Toaster manually works well, however.
Regarding optimising the same mesh with PyFFI multiple times, I don't think it should cause any problems. An older version of PyFFI famously had a problem with broken collision, and running an already broken mesh through newer versions probably won't fix that; for that you'd have to start over with an unmodified mesh. Using a recent PyFFI version multiple times on a mesh should be fine; if the mesh is already optimised it just shouldn't do anything. It probably also doesn't do any good though. PyFFI versions before 2.2.2 had a bug where you sometimes had to run it twice to catch all optimisation possibilites, but that should be fixed now.
Lastly, regarding the skip list, I've found that it's quite conservative. PyFFI can handle the Night Mother statue just fine now, for example, and it's also safe to optimise closed-face helmets, since they don't have an associated EGM file. Even some meshes with hair in the name are safe, as long as they are for creatures, such as boars, trolls and zombies, rather than for characters.
It seems I stumbled across the best solution, at least for Oblivion, by sticking with 2.2.2 and 3.3.x without being aware of it. Your comment regarding texture path substitutions does worry me, since (besides the issue with vanilla assets that have a leading space in the name) that is one of the spells that executes in the automated batch. But I've not noticed any missing (i.e. purple) textures to date, so hopefully that means all is well.
Interesting, too, that the Night Mother statue and other meshes long on the "do not touch" list may be safe now. I may experiment with those. I also hadn't noticed that closed helmets lack EGMs but it makes sense that they wouldn't need them.
Thanks again!
Between PyFFI, Ordenador, TES4LodGen and LandMagic, I think I'm in pretty good shape on the meshes and texture end now. I've considered running all the voice MP3s through a volume-rebalancing algorithm (MP3Gain v2, specifically), but that requires the assumption that every file is supposed to be at the same volume. Which isn't true. So there's probably no way to automate it.
I've tried to fix meshes of the mods in your list and Cobl seems to break. I CTD in player houses after using Sniff on its meshes (not all of them, just the affected ones). I don't really want to dig into this so I just left Cobl untouched for now.