NorthernUI's been out for nearly three years now, and I guess I just want to write down some reflections on its development -- challenges I ran into, things I could've and should've done better, and the like. I don't want to use the term "post-mortem" because I don't want to permanently write off ever returning to maintain NorthernUI, but I suppose it'll be that general kind of write-up.


Scope and feature creep

This is absolutely the biggest and worst issue with NorthernUI. We're not dealing with Oblivion Reloaded levels of creep here, in any sense of the term, but NorthernUI does too many things at once and absolutely should've been multiple mods.

Having the gamepad support exist in the same DLL as everything else is definitely my biggest regret and the main source of feature creep. The only reason I implemented gamepad support at all was because Xpadder/AntiMicro wasn't satisfactory for navigating the game's menus; the only way to smoothly handle switching between gameplay and menus is to actually have built-in gamepad support. Unfortunately, I was unfamiliar with the Windows APIs for working with DLLs; I taught myself C++ and x86 assembly by making SKSE plug-ins, and SKSE, unlike OBSE, offers a simple mechanism for sending messages between plug-ins, so I never needed to learn the Windows APIs. (Essentially, when it comes to C++, I learned a lot of the hardest and most niche stuff first, and a lot of the basics last.) I didn't know how to pass messages between OBSE plug-ins, so to my thinking, anything that needed to access gamepad state needed to be part of the same DLL that was adding gamepad support in the first place. This is how features like enhanced movement and camera controls ended up in what was supposed to just be a UI reskin.

For anyone who's wondering, the right approach would've been to make the gamepad support a fully separate OBSE DLL, and have it "export" functions to be called by other DLLs. Then, any interested OBSE DLL could use GetModuleHandle both to check if the gamepad DLL is loaded and to gain access to its exported functions via GetProcAddress. With a system like that in place, features like the enhanced movement and camera behavior could have been placed in yet another DLL, keeping things neatly separated.

(Oh, speaking of which: do you find Oblivion's third-person camera uncomfortable and disorienting? Do you hate how strafing tries to turn the camera to the side to "peek" in the direction you're moving, such that your movement desynchs from the camera? Sick of the wild swings the camera takes every time you step over a curb? Set NorthernUI's "Camera Inertia" option to "Disabled" for a less vertigo-inducing experience. I literally forgot that I implemented that and only rediscovered it today, and holy heck does it make this game more tolerable! I'm so grateful that the camera inertia was gone by Skyrim. I'm still going to leave the inertia on (but with fixes) as the default setting, because Oblivion is Oblivion and should be Oblivion by default.)

If it had occurred to me to rely on this kind of DLL communication, I could also have allowed NorthernUI to provide an API for its options menu: I could've defined "option handler" classes with virtual member functions, and allowed NorthernUI to receive pointers to instances of these. Other DLLs could've then defined their own subclasses which override these functions, to specify the option's widget type (checkbox, slider, etc.), supply a value, react to the value being changed, and so on. This idea could also be taken further to allow other DLLs to insert nested menu structures. If I had separated my enhanced movement and camera patches into their own DLL, they could've used this framework; it could also be useful to the handful of other actively maintained DLL mods out there. (I did at one point consider the logistics of allowing scripts to add menu items to the options menu, but I couldn't think of a good API design for them.)


Control schemes

NorthernUI includes multiple pre-defined control schemes, and allows you to define your own as well. I don't think I did a good job of presenting the feature, but I think I did a decent enough job of implementing the feature. It was inspired by the approach to controls taken by games like Halo, where you have a wide variety of control schemes that emphasize different actions and abilities, like the iconic Bumper Jumper which is intended to let you jump and melee without having to take your thumbs off of the aim thumbstick. I wanted to offer some good pre-defined control schemes, but also let you create your own. IIRC Borderlands 2, by contrast, offers a few predefined control schemes and lets you make just one custom scheme, which is more limited but perhaps more intuitive than what I've done.

Looking back, it would've been nice if I could've actually had a graphic of an Xbox controller, with labels tied to each of the buttons to literally let you see the different layouts. As it stands, though, I think the look of the customization menu is on par with the rest of Oblivion's menus.

One thing that I'm pretty proud of is the one-handed control schemes. These were pretty much directly and entirely inspired by halfcoordinated's speedrun of NieR:Automata at Games Done Quick. I tried to design each control scheme to ensure that the most critical gameplay functions were quick and easy to access, though I'm not nearly skilled enough at games to try them out myself. I can barely manage with two hands.


Compatibility challenges

One thing that I didn't see coming at all was the number of compatibility issues that would come up when making a UI reskin. It turns out that an absolute ton of scripted mods rely on access to the UI to influence game state and features, and the script functions that these mods rely on are spectacularly brittle. Both OBSE and MenuQue's UI commands require that UI elements be accessed by a fully-qualified path. This means that if I change how elements are nested, or if I change the name of any element in a nested hierarchy, then those commands immediately stop working.

A few of these mods' authors included code to automatically detect DarNified UI and use different element paths for it. I recall one author actually embedding the element paths into a secondary file as variables, such that they could in theory add new paths very easily. Most of these mods' authors don't seem to have planned ahead for new UI overhauls at all, however... and they've been long-gone for years. Several of the mods in question use closed permissions, so providing specific patches for them wasn't necessarily an option (plus Oblivion embeds its scripts directly into ESP files, so any patches would've burned load order slots).

I also don't play Oblivion regularly, and never did. I got the game on sale years ago, and decided to play it casually while dealing with a real-life situation that got in the way of working on Skyrim projects. That turned into me disliking Oblivion's UI strongly enough to want to adapt it to something more familiar, which turned into... all this. My experience as a player involved grabbing the most popular graphics mods and one or two stability fixes and then diving in. The kinds of mods that use scripts to tamper with the UI are exactly the kinds of mods that I, as an ultra-casual player (and now pretty much not a player at all), would never have found on my own, so I was completely blindsided by the compatibility struggles.

What I ended up having to do in several cases was modify the UI to provide invisible elements with the fully-qualified paths that these mods expected, and then have the UI copy data to and/or from these dummy elements; this in turn required reading the mod's scripts closely to figure out what it needed. (This was fairly difficult for me just because I'm not used to Oblivion's scripting language, and my experiences with it have seldom been pleasant: it's clunky to edit, clunky to compile, often compiles incorrectly...) In other cases, it was easiest and cleanest to just take the entire mod concept and implement it in NorthernUI as a feature.


No vectors, rasters only, Final Destination

Skyrim's UI uses vector graphics by way of Scaleform, a framework that essentially allows you to run Adobe Flash inside of a video game. (About half of you reading this are going to say, "Wow. Bethesda would do something that jank." It's not just them.) This makes UI modding very inaccessible, because you need a particular version of a particular toolset intended for business customers and priced for business customers -- and that's without even getting into the fact that Scaleform was discontinued a couple years ago. There is one obvious benefit, though: you get all the benefits of using Adobe Flash, namely vector graphics that can be displayed with perfectly smooth linework at any scale, and also a variant on JavaScript if you're into that sort of thing.

By contrast, Oblivion's UI is defined using XML and DDS texture files. The game takes that XML and renders the elements inside in 3D, as a series of dynamically-constructed quads with textures mapped to them. This means that all UI graphics, including fonts, are game textures, and it's why the UI is affected by Oblivion's different graphics settings -- including the FOV option which, if changed, will completely destroy all the careful alignments Bethesda used to get the UI to show up properly. I mean, hell, the file format for Oblivion's fonts literally defines each glyph in terms of size, alignment, and texture UV coordinates. This kind of thing has a number of consequences for NorthernUI -- some of which were very unexpected.

Probably the strangest issue I ran into is that if any texture smoothing is applied to the game's mouse cursor (which happens by default), then every UI texture except for the cursor will end up absolutely mutilated. Imagine scaling a graphic down by half, scaling it back up with nearest-neighbor, and then blurring the hell out of it. You don't really notice it with Oblivion's vanilla UI because the skeuomorphic design that Bethesda went with isn't really affected by it so much, but when you have a style that revolves around clean, crisp lines, things get very bad very quickly. I ended up having to hack part of the engine to specifically exclude the cursor from any texture smoothing, so it renders with jagged lines but everything else looks fine.

Apart from that is just design issues. Raster graphics don't scale cleanly, and because Oblivion adapts its UI to your aspect ratio, it's always going to be scaled at least a little bit. The font format doesn't lend itself to scaling very well either; in theory, you could create an ultra-high-definition font texture and then set the glyphs up to define small quads, allowing a small-print font to render crisply and cleanly even at very high resolutions. In practice, something strange seems to happen to the UVs, so the font texture doesn't get cut properly and the letters get mangled.

If there was one thing I really wish I could've done, it's add some sort of support for vector graphics to the engine. If we could render SVGs and perhaps a real font format, then it would've become a lot easier to produce a high-quality UI at any resolution, and it would've been possible to offer a wider variety of options for things like UI scaling and (in particular) font size, the latter of which would've been great as an accessibility option. NorthernUI as-is probably looks fine to you, but I can't help but notice little inconsistencies like how the compass linework (at its default size) is noticeably thicker than the linework for the health/magicka/stamina meters, because scaling the compass down enough to match would likely have blurred the linework all to hell at most resolutions.

At one point, while I was struggling to figure out that texture smoothing issue, I got so desperate that I actually 3D modeled a single graphic from Skyrim in Blender -- specifically, the "up" endcap for scrollbars. Oblivion lets you embed NIFs directly into the UI (this is used to power anything that uses complex animations, including the lockpicking and persuasion mini-games), so I tried to use that as a cheap way to do "vector graphics." But remember what I said before: the UI uses the same render settings as the game world. (The UI also strips most shader settings off of the NIFs it loads, but it was easy to stop that with a patch.) My scrollbar endcap would look perfectly smooth if you ran the entire game with strong anti-aliasing, but IIRC Oblivion uses MSAA so that could have a heavy performance impact. Without anti-aliasing, the endcap was jagged and totally unusable. What a shame.

The 3D endcap as shown in NifSkope, with the grey portion selected and its polygon edges visible.
Mostly this was the 3D equivalent of a traceover, but IIRC as a shortcut I sliced a disc in half for the rounded base.

The original box corner graphic on the left, at 200% zoom, and the current box corner graphic on the right, at normal zoom.
Note the quirks in the linework on the right, mimicked from the original vector art in Skyrim.

The original compass graphic on the left, at 300% zoom, and the current compass graphic on the right, at normal zoom.
Note how the newer graphic is more on-model, both in terms of proportions and angles, and in terms of where the knotwork overlaps itself.