• 12 February 2014

    Integrated support and separation for other language and "translation mods"

    posted by Dark0ne Game News
    Files that are written in languages other than English and "translation" mods (authors who have translated an English mod in to another language) have always caused a bit of a pickle for users on the Nexus due to the fact they're mixed in with all the other mods in one big soup. If you're an English-only speaker it can be a bit of a pain having to navigate past numerous mods written in different languages, or indeed multiple different translations for the same mod. Similarly if you're looking for a translation of a popular mod in your language that, too, can be difficult and inefficient. Some popular mods can have many different translations. If you search for Immersive Armors, for example, which has 12 translation mods made for it, it can actually be a bit hard to find the main mod in between all the translations (you typically need to look for the file with the highest number of downloads).

    We've always been a bit apprehensive about files uploaded in languages other than English. They're harder to moderate and they can, as mentioned above, cause a lot of bloat in the file database. Today's update is supposed to help in this regard. We've released a foundation update to the sites that is supposed to help better separate the multi-language content here on the Nexus and allow you, the user, to choose which content you see and which content you don't see, hopefully making browsing the file databases that little bit easier.

    So, where to start?

    On the file uploading side of things, authors will notice that we've added a few more fields to Step 2 (or the first step if you aren't picking a game for your mod first). These new fields are right at the top so they shouldn't be hard to miss. You now specify whether your file is a "Normal file/mod" or whether it is a "Translation of a file/mod that already exists". Pretty straight forward. If it's not a translation, it's a normal file. Next, you specify the language of your mod. There's a drop-down that covers the main ones, and if the language your file is for isn't listed you simply select the "Other" option and state it in the file name.

    For normal files that's all there is to it. However, if your file is a translation there's an extra field to fill in. Specifically, what file your file is a translation of. We've stipulated a new rule with this addition, which is quite important, that we are now only allowing translations of mods actually on the Nexus. This goes hand-in-hand with the new functionality of this update. We need to be able to link the translation to a file on our database. Begin entering the name of the file and you should be presented with a list of mods that fit what you've entered. Click the correct file from the list and that's it.

    On the file authoring side that's the only change; 2 or 3 new fields to fill in.

    The changes to file pages depends on what sort of file it is. If your file is a normal file (and not a translation) then there is no change at all unless someone has created a translation for your mod. If there is a translation for your file then a new small button will show in the "Actions" tab (below your file stats) labelled "Translations". Clicking this button will show a pop-up with the translations available for your mod, the language of the translation and who made the translation. If your file has no translations then this new button will not show on your page.

    If your file is a translation then there's two changes to a normal file page. Next to where the "Download (NMM)" or "Download (Manual)" button would normally show is a new button labelled "Original file". This links back to the file which your translation is based on. Similarly, if someone attempts to endorse your translation then an additional step is added to the endorsement process. This extra step asks the user if they would like to endorse both your translation and the original file, or just your translation and not the original file. As a mod author this might sound a bit odd to you (or, hopefully, it doesn't), "But Dark0ne, how can someone like a translation but not like the original file enough to endorse it?". Well, simply put, someone might not like your file enough to endorse it. They might still use your file and they might want to thank the person who bothered to do a decent translation of your file. It's not beyond the realms of possibility and it's important that users are given the choice so it doesn't demean the endorsement system.

    Those are the only changes on the file pages. Next, we've also made changes to the file searching by adding an extra "Language" option to the list of variables you can sort through on the right-hand side.

    First of all it's important to note that translations will no longer show by default in the search results. If you're looking for translations you should tick the "Only show translations" box, which does exactly what it says it does. Next up, you can now specifically choose to only see results for file results in your specific language. If you select "English" you'll only see files marked as English. If you select "Portuguese" you'll only see files marked as Portuguese, and so on. This is stipulated by the drop-down menu in Step 2 when first adding a file or in "Edit Attributes" if you've already uploaded a file before this update. If you do not update this information then your mod will not show when a language is picked by the user. Which brings me on to the next point...

    This change is opt-in rather than opt-out. Or better put, it was not/is not possible to create a script which will automatically convert all translation files already in the database in to the proper format. Similarly, we can't create a script which will automatically update the current files in the database with the correct language tag. I'll be changing files as and when I have the time and as and when I see them, but we're relying on the actual authors of their files to update them accordingly. And just a quick tip, if you don't update your file's language then you won't show in the results for that language, e.g. right now I notice a few people have already changed their language to English before I've posted this article. Their files will show when a user has selected "English", your file, if you haven't changed it, won't. So you'll be missing out on exposure, if that bothers you.

    As I mentioned earlier this is a foundation update, which means it's an early release which will probably be updated or changed based on the feedback you provide on the functionality. If you think it can be changed to work better then let me know and we'll have a think about it. The design document for this update was posted in the mod author forums for scrutiny with no complaints so I'm hoping this change will be welcomed.
  • 10 February 2014

    NMM 0.47.2 bugfix update released

    posted by Dark0ne Site News
    Update - 11th Feb
    We have added another hot fix, up to 0.47.3, which has patched in a few more fixes.

    We have just released a bug fix 0.47.2 update to NMM that fixes a few bugs that were introduced with 0.47.0. Here's the changelog:

    • New Feature: Better feedback on login issues.
    • Bugfix: Add mod from file asking the user to login.
    • Bugfix: Download speed messing up after a resume.
    • Bugfix: Download from URL button disabled while offline.
    • Bugfix: Cancelling a paused download after a program restart didn’t remove partial files.

    A few users have reported getting the "Cannot connect to login servers" error even after updating to 0.47.1. We've added a bit more detail to the error so if you tell us what error you're getting now we'll be able to know a bit more about the issue. For most people the issue is your firewall blocking NMM from connecting to the login server.

    Each time the NMM executable changes or gets updated your firewall picks up (or should pick up) this change and change its settings/prompt you about it accordingly. I know this happens with ZoneAlarm, for instance. You need to update your firewall accordingly so NMM can use the internet else, obviously, NMM cannot login and use any online services.
  • 09 February 2014

    Skyrim Mod Sanctuary 84 : Skyrim Memory Patch (Fix freezes and crashes)

    posted by Gopher Game News

    Many of you will have heard of the Skyrim Memory Patch, but what does it do, and is it something you need? In this video I discuss the problems this patch attempts to fix, and will show you how to test it out.

    Further information:
    Skyrim Memory Patch
    uGridsToLoad information
    SKSE Installation Video

    Download Links:
    SKSE - Skyrim Script Extender
    SSME - Skyrim Startup Memory Editor
    Memory Blocks Log
  • 05 February 2014

    NMM 0.47.0 "forced" update to fix the download issues

    posted by Dark0ne Site News
    Over the past few weeks the NMM crew have been working tirelessly on a new version of NMM. The main focus of this version is to dramatically improve, if not altogether eradicate the incomplete download issues that have plagued NMM for the past month or more and to also reduce the rather substantial requirements and strains that NMM is placing on our server-side infrastructure.

    On 0.46.0 I was getting a 25% completion ratio on downloads. That is to say, if I started 10 downloads, 2-3 would complete as expected, 7-8 would fail. Obviously that's not good at all. We know that, you know that. With 0.47.0 the proof is in the pudding, while writing this update I started 50 downloads in NMM, by the end of writing all 50 had finished with a 100% completion ratio. Much better.

    So what have we changed?

    On the back-end, non-NMM side of things we're fixing the bugs that were/are causing the "file does not exist with this ID" error, which were also causing the issues with NMM. However, on top of that, NMM has been recoded to now try the same server 3 times, and if all 3 attempts fail then NMM will then try to download from another server entirely. Rinse, repeat until a server that actually works for you is found. This has probably helped to rectify 95% of all the original download issues that existed.

    We've changed the way in which NMM picks the servers it uses based on your NMM option settings. Lots of people don't seem to know that if you go in to NMM's options you can pick your closest location. This doesn't 100% guarantee NMM will always use the server's near to you, but it will always try them first before moving on to servers that will actually work for you or won't be overloaded. For Premium Members, you can now pick the specific Premium server you would prefer to download from. Once again, NMM will always check that server first, but if that server is down, overloaded (shouldn't be for Premium) or struggling then NMM will automatically find a different server instead.

    As mentioned in my previous news article on the subject, we've also changed the amount of concurrent downloads you can have running at any one time. With the rise of sites that provide easy compatible lists that allow you to download mods more quickly (because you trust the source and don't read all the file descriptions) we've seen lots of users trying to download 50, 100, even 200 files at the same time in NMM. This obviously isn't good for the servers so we've coded in a queuing system that will only allow you to download a certain amount of files at a time. For non-Premium users that number is 5, for Premium users that number is 10. Now, if you try and download 50 files at once then all 50 files will be added to NMM's download list, but only 5 will download at once. Once a download is finished, another is automatically started from the queue until all the downloads are done. This should help to alleviate a lot of pressure on our file servers and ensure that everyone is using the service fairly. This also pre-empts a planned future functionality to allow users to share their mod playlists and collections with others.

    In the same vein we've changed the way NMM starts and connects to the internet. Before this new update all versions of NMM attempted to login to our servers before NMM would fully load. However, this seems inefficient considering there are many times when you want to use NMM; to modify your load order, to change your installed mods, and so on, where you don't need to be connected to the site. On-top of that, each time you started NMM it was requesting various bits of information from our servers that you didn't necessarily need. For example, the "Latest version" information. To save our servers a crap-ton of stress we've now made NMM start offline for everyone. NMM will no longer automatically connect at the start and check for the latest version information. Instead, if you want to check for the latest versions of files you should start NMM and click the green arrow icon to the left of the "mods" tab, labelled "check for new mod versions" when you hover over it. This not only benefits us, from a server perspective, but also benefits you, as NMM should now start faster for you (as it doesn't have to wait to connect) enabling you to get on and do what you want to do faster.

    We've added a little buddy in the bottom left hand corner of NMM to let you know if NMM is offline or online. If NMM is in "offline" mode then our little buddy is red. Clicking him when he's red will either automatically put you online (if your login details are saved) or prompt you for your Nexus username and password. If the little buddy is green then NMM is in online mode, and clicking him will log you out. If NMM is in offline mode and you try and do something that requires NMM to be in online mode, like checking for new mod versions or downloading a file, then NMM will, once again, either automatically log you in without prompting if you've saved your login details, or ask you for your login details if you haven't saved your details.

    While I understand that lots of people like to check for the latest versions of their mods we believe this update, while inconveniencing you with one-additional click if you want to check for new versions, will be worth that inconvenience.

    Because this update fixes a major issue with NMM that is being reported a lot daily, and because we hope it is going to free up a substantial amount of server resources, 0.47.0 is going to be a "forced" update. That is, any versions before 0.47.0 will not be able to go online. We've updated our process now so that if you are using an incompatible version of NMM it should now tell you as much, and tell you that you need to update NMM in order for it to be able to login again.

    Once again, the update process for NMM is very simple. Indeed, it seems to freak people out when they don't update from "within" NMM. Actually, all NMM does is download the latest update from the site for you. You still have to click through the install process. To "manually" update NMM simply go to our NMM download page and download it. Once you run the installer you've hit the exact same step you'd have hit if you'd clicked "yes" to updating from within NMM. All you need to do is ensure that the paths in the installer to where you have NMM and your mods installed are correct. That's it. All your mods will still be where, and how, you left them.
  • 31 January 2014

    Required files notification and comment download history update

    posted by Dark0ne Site News
    We've added a relatively small update to the sites today that revolves around the "Required files" system in the file database. When an author adds their file to the site via our file wizard one of the steps involves adding any files that the author's file needs in order to work. This information is then displayed in the rather small, out-of-the-way button labeled "Required" on the file page. Typically authors also put any file requirements in their descriptions but, as I'm sure any file author would be willing to confirm, some users seem incapable of actually reading descriptions, installation instructions or indeed anything pertinent to do with the file before they add the file to their game. Guess what some of these users then do when they can't get the file to work or it breaks their game? That's right, they head off to the file's comments to give the author a piece of their mind or ask for help when all the information they need is right there, on the file page! It's annoying.

    We've made two changes to help better inform downloaders about the file's requirements, and also to inform the mod author as to whether you've actually downloaded those requirements when you comment on their file page.

    As an author, if you've specified requirements for your file you will then be able to tick a checkbox when uploading your files that will turn on a notification window for anyone who downloads that file. The notification window will inform the user that your file requires other files in order to work, it'll list those files, and then require the downloader to click a button in order to get to the fileserver list or download your file via NMM.

    Please note that we've made this an opt-out feature, rather than opt-in. That means if you've specified required files on your files then this notification window will be turned on as default on all your uploads for that file. If you want to turn off this notification window you need to edit your uploaded files and untick the checkbox.

    Back when we updated our commenting system a short while back we added the functionality for mod authors to see what, if any, files the users who were commenting had downloaded from your file page. We've expanded this functionality now to include the user's download history of the required files too. You'll now be able to see if a user has downloaded the required files you list, as well as your files, which should help you to more quickly troubleshoot whether the user has actually downloaded the files necessary in order for your file to work.

    Fingers crossed we should have another update today in regards to an update to NMM to fix the downloading issues. We've been testing it the past few days but there's still a couple of niggling bugs that need to be squashed before it's safe to release. Good news is, I go from a 25%-50% chance of a download working to 100%. Shock.
  • 28 January 2014

    Skyrim ENB Mods 5 - Phinix Natural ENB

    posted by Gopher Mod News

    This time I take a look at Phinix Natural ENB and see how it looks and how it performs.

    Download Links:
    Phinix Natural ENB
    Dark Dungeons for ENB
  • 20 January 2014

    Skyrim Mod Sanctuary 83 : Equipping Overhaul

    posted by Gopher Mod News

    An overhaul of the system for equipping your gear in Skyrim that can be customized to give you only the features you need.

    Thumbnail is courtesy of turningworlds.

    Download Link : Equipping Overhaul
  • 08 January 2014

    Downloading and file serving changes

    posted by Dark0ne Site News
    Happily and thankfully a week after I wrote my 2,600 word End of Year review/essay we managed to stabilize our new server setup and run through some quick but efficient code updates that have enabled us to run the sites almost down-time free over the past week. No database desyncs, no maintenance windows, just a few times where we've taken the pages down for a few minutes to perform code updates. Paint me f'ing happy on that. And vindicated in my own eyes. No doubt we've still got work to do, but the important thing is that things are way better now.

    In the same article I also mentioned that during January we would be decommissioning all 21 of our file servers (2 static content servers, 15 file servers, 4 Premium file servers) and replacing them with new servers. The old file servers were coming to the end of their life-line and running out of hard-drive space fast so we needed to upgrade and update the whole line up. We're now replacing those 21 file servers with 26 new file servers (4 static content servers, 20 file servers, 2 Premium file servers with probably more Premium servers to come). As you can imagine that's quite a job; we've got to set up each server with our custom server modules and configurations as well as transfer the 4TB or so of current files there are on to every single one of those servers (so we've got to transfer about 100TB of data over the next month). It's time consuming to be sure. We're currently setting up the first new servers which requires these new servers sync from the old ones, this is causing some issues with the network which is causing some downloads to fail (more than usual) or not start at all. Once we've got these first new servers setup we can sync the files between all the new servers, from the new servers themselves, thus reducing the load on the old setup in the mean-time.

    I know there's been some issues with downloading reliably through NMM recently, especially when downloads don't start at all. In those instances it's important (for your own sake, not mine) that you know how to do the simplest of things: how to add mods to NMM without using the "download with manager" button. It's so simple I might as well just write it here:

    • Instead of clicking "Download with manager" click "download manually" on the file you want to download.
    • A window will pop up with a list of file servers to download from. Pick a file server. Typically the fastest server will be the one with the least users, not the one that's closest to you.
    • Your download should start within 1-30 seconds. Let the download finish.
    • When your download has finished, open NMM.
    • Open the folder where your file downloaded to.
    • Drag the file in to the "mods" window in NMM.
    • OR: in the "mods" window, click the button that looks like a jigsaw piece with a plus (+) symbol next to it and navigate to your file on your HDD.
    • NMM will add your mod just as if you'd used the "download with manager" button.

    That's it. If you download from the Nexus and don't change the filename then NMM will recognise the file and the file will work exactly as if you had downloaded it through NMM itself. There seems to be some myth that if you download manually then NMM won't show you any new mod versions. That's wrong/a lie. NMM cannot show you mod versions for mods not downloaded via the Nexus (so if you download a mod from the Workshop and add it to NMM then NMM won't show you any new mod updates/versions for that file), hopefully for very obvious reasons, but it will show you updates for any Nexus mods whether you downloaded the file directly through NMM or manually via the site.

    At times like these you will find it easier and quicker to download your files manually and then add them to NMM rather than using the "download with manager" button and have it failing on you a lot.

    While we're working on the file servers we're directing a bit of attention to how NMM handles downloads, and especially downloads that fail. Right now NMM will try and find a working server that fits your download location preferences as well as possible. However if that server fails then it doesn't automatically try another server, it just fails. We're trying to add more fail safes in to the program so you're left with no failed connections.

    Similarly we're having to implement a download queuing system based on feedback I've personally been getting from a few...interesting individuals. With the ever increasing interest in projects like Skyrim GEMS and S.T.E.P. there's a lot of users who are downloading large quantities of mods all at once. I'm getting support tickets from users wondering why they're struggling to download 200 mods all at the same time in NMM. Hopefully I don't need to tell you why downloading 200 files at the same time isn't good for the servers (or respectful to the other users of the site). Now this is partially our fault as we haven't hard-coded any limits in to NMM in this regard. NMM will let you download as many files as you want. They probably won't all work, but it'll let you do it. We've got hard-coded limits on our servers (which I believe are 16 connections for normal users and 64 for Premium, with an increase for Premium as they can use 4 threads, 4 multiplied by 16 is 64). With currently 15 file servers that's a theoretical maximum of 240 concurrent downloads, but that works on the assumption that NMM will pick servers that you aren't already maxing out, which it won't. But I digress, trying to download 200 mods all at once is really stupid and us not having hard-coded limits in the program is equally stupid. As such we are going to be introducing a download queuing system. Normal members will be able to download 5 files concurrently, Premium members will be able to download 10 files concurrently. You can add as many files as you want to your download list but only 5 (or 10) will download at any one time. Once a download finishes a new download in the queue will start automatically. Such functionality is commonplace on many sites and will enable us to provide a more stable and fair service to all users.

    We hope to get this new NMM version out to coincide with the completion of our transition over to the new file servers. It's likely this version will be another "forced" update, in that you'll need to update to this version in order to be able to use NMM online, because it's updating and fixing things that are the cause of issues on the network. I'll have a word with the programmers to see if we can update the error you receive when trying to login to an unsupported version of NMM rather than the generic error that is currently used.

    In the meantime I hope you're enjoying a much smoother site experience, even if the downloads are currently a little iffy. The important thing is: we're getting there. Like I said we would.
  • 06 January 2014

    Nexus Mod Manager plans, answers and if you're a coder, we want your help

    posted by Dark0ne Site News
    The aim of this article is to answer a few regularly brought up questions and concerns in regards to the Nexus Mod Manager, talk about the plan for NMM this year as well as talk about how you yourselves can help us.

    I think one of the biggest questions that gets thrown around about the Nexus Mod Manager is why, after more than 2 years, it is still in open beta. Some people seem to find this rather insulting and claim we're using it as a cop out for not following a strict stable/beta release structure. I can tell you right now, without any sense of shame, that's pretty much exactly why NMM is still in open beta.

    We don't hide this either, it's not as though we claim NMM is some super stable, magical, bug free piece of software that's going to be the solution to all your modding issues. It's really not and we don't claim it is. When we say that, "We cannot stress enough that we are still in the beta stage of NMM. NMM is in a very good state and will be usable by nearly everyone, but Beta stages are typically used for testing and bug fixing, and some of you will find bugs." on the NMM download page and put "open beta" on our pages and within the program itself, that isn't just some blurb we put up to fill column inches, we actually mean it. Even the versions of NMM that are considered "stable" aren't very stable, and to put a "stable" moniker on any current version of NMM would be misleading for new users. No. When you download NMM, when you use NMM, you do so being fully and properly informed that NMM is still open beta and is still susceptible to all the foibles that comes with that. If you don't like that, if you're not comfortable with that, then absolutely nothing is keeping you using NMM over any other mod manager out there (or, manually installing the mods yourselves!). This is how we've chosen to do this as it's making our jobs a lot easier at this time. Once we get a 1.0 version of NMM out we will then most definitely change our structure to ensure we follow a more Linux oriented style of NMM releases (e.g. stable release/bleeding edge seperate releases). Until then don't act like NMM is anything more than in open beta, because it isn't, and if you have the idea that it is then you didn't get it from us.

    The reason why we're not just slapping a 1.0 version on NMM and moving into a more stable release structure is because NMM is not complete. I wouldn't say it's "no where near complete" but I'd say there's still a long way to go. And I think perhaps "complete" is the wrong word to use, I instead mean "complete to the point of being happy to put a 1.0 moniker on the program" because NMM, in all it's cliche glory, will never be complete, as we'll always be adding to it and upgrading it even after our 1.0 feature list is implemented.

    When I first contacted kaburke, a developer of the Oblivion Mod Manager (OBMM) and Fallout Mod Manager (FOMM), about coding NMM we wrote out a rather informal design document that contained a number of features that NMM would need to contain before it would be considered 1.0. Over the years there's been a bit of feature creep here and there, but to this day several of these features are yet to be finished. So you're probably asking, what features are these?

    • Stability. Naturally I'm not going to release a piece of software that is still struggling to get online half the time.
    • Bugs. As bug free as is feasible considering the myriad of hardware and software combinations out there.
    • Documentation. Proper, decent, detailed documentation on how to use NMM.
    • Modern UI interface. We commissioned work on a new design for NMM towards the beginning of last year and I showed you some of the work that had been done at the time. A 1.0 version of NMM will not look the same as it does now.
    • Mod profiling. More on this below.
    • Mod packaging. A piece of accompanying software for mod authors that will help them package their files into a single, open, proprietary file (e.g. a .nex file) that will ensure their mod is NMM compatible and also help with setting up custom/scripted installers for use within NMM while also remaining open so anyone with a zip program can open/use/extract the files manually if they so wish.

    Some of these features are major additions to NMM that require extensive testing, the sort of extensive testing that can only be achieved if the majority, rather than a small test pool of users, are helping us to test the program. So we rely on you to help us.

    It's now been over 2 years since we released NMM and things are going very slowly. It's not hard to see and, once again, I don't hide that fact. So what's taking so long? It's simply the sheer scope of the project at hand. Take mod profiling for example. We started work on mod profiling back in the middle of March 2013. That's 9 months ago, and it's still not done.

    Which leads to the next regularly asked question; why is it taking so long when programs like Mod Organizer have had it for yonks now and it was seemingly coded in a shorter time? Simply put and specific to mod profiling, we're having to do it in a completely different way because the scope of NMM, which has to support modding for multiple different game engines and not just GameBryo, is much larger. As such we've had to use different methods that aren't just specific to one engine. The result will be a mod profiling system that will not only work for all games but will also continue to work if game developers make some changes to the way modding in their games is handled. And herein lies the issue. Mod Managers developed for specific games or specific engines are always going to be at an advantage in some regards to NMM which is trying to create a platform from which any and all games can potentially be modded. That doesn't mean that NMM can't have powerful features unique to individual games it supports, but while NMM has a framework from which we work from, engine specific mod managers are more free to go off in any direct and do anything without being confined by an encompassing framework which often requires more time to develop for. The framework is what allows us to support all games in one piece of software but it also presents additional barriers that need to be overcome compared to a piece of software developed specifically for a single game engine.

    It's why when certain high-and-mighty users of other mod managers feel the need to come take a dump on the progress of NMM I dismiss the criticism for the ignorance it exudes. Thankfully, and much to the credit of Mod Organizer in particular, I know the author is actually a rather stand-up individual who's done everything in his coding power to ensure that the NMM services he uses aren't placing an undue burden on our servers. He hasn't caused a hassle at all (even when services have been changed or taken down) and has responded to any questions asked in great manner. So when such individuals come and act like self-righteous asses on the topic of NMM I'm able to form a clear dividing line between these minority users of MO and Mod Organizer itself, and its author, which are both great.

    Lest people forget, the services used to allow users to check for new mod versions or download straight from the site in to NMM or MO or any other program are currently open and free for anyone to use. We've kept these services open because we didn't want to be those types of developers who make everything open source apart from the one thing other programs might need in order to be on a level footing. And to clarify, NMM is open source, our web services are not (they're completely different entities). Nothing is stopping us from closing the web services to anything other than NMM in the future, and the main reason we haven't is because (a) no one has given us a reason to close them and (b) the people who are using them are using them well and in a morally sound way. In the future we'd like to create an API to be able to better control who can use the services, and extend the services on offer. I know that the S.T.E.P. crew, for example, have been wanting a way to allow more than 1 mod to be downloaded from a single link to make the process simpler, and that's something we want to look into, but considering the time it's taking us to develop things right now I don't think it's going to be happening any time soon.

    Which brings me on to my next topic. Help. While we've been going through the recent site issues a lot of people have been asking "How can I help?". On the website end there really isn't much anyone can do except not mash F5 twenty times a second if something goes wrong, and perhaps become a Premium Member. But with NMM things are completely different. You can actually help, and you can pay with time, not money (and maybe even earn some money).

    When we released NMM we made it completely open source. That means anyone can go to our SourceForge page for the NMM project, download all the source code and see exactly what NMM is doing and how it's doing it. I wanted to do this for a number of reasons:

    • It sets us apart from other modding networks who have mod managers but haven't made them open source to ensure they maintain some form of "competitive edge" in a niche area of the gaming industry that I don't think should be competitive at all.
    • It enables us to be completely transparent about what NMM is doing on your PC. If you're worried it's doing anything bad, malicious or nefarious, intentionally or otherwise, then you can either look for this yourself or someone else can find it and let others know. It enables us to remain above-board with you.
    • My stance on mod publishing is well known among mod authors. I think all mod authors should release their mods with an open and free license to allow users to fix bugs and add or change features based on personal preference, and that these fixes and changes should be allowed to be shared on the premise that the original mod is still required. That's my feelings on the matter. I couldn't in good conscious release a mod manager that didn't follow that philosophy. NMM is open source which means you can add to it, change it and publish it anywhere and I can't do anything about it.
    • It means anyone and everyone can help with the development of NMM.

    Unfortunately that last point has never really happened. While a few people have expressed an interest in helping with NMM it never turns into anything. This community has a lot of talent within it across a broad range of skillsets. There are plenty of programmers out there and I would like to harness that potential to help us with NMM.

    When you write on the forums that you can't believe NMM doesn't have feature X yet, or that bug Y has not been fixed, you need to realise that you yourself can add feature X, and fix bug Y, and in not doing so you've, at least in my eyes, accepted that we will (or won't) add feature X and fix bug Y in our own time. If you don't like that then crack open your favourite coding program and do it yourself and help out millions of other users in the process.

    We recently did a trial run with a developer from the X:Rebirth community who coded the integration of X:Rebirth in to NMM which was recently added. It went brilliantly. So much so I'm now talking with another developer from the Starbound community to get NMM integrated with Starbound modding. I think this is an untapped resource in the community that could be put to good use if done right and with the proper incentive.

    I was planning to hire an additional .NET programmer this year to help the 2 we currently have already working on NMM. However what I'd like to do is run a little experiment within the community over the new few months to see if that money could be better spent working with multiple individuals on a project by project basis. Anything from some simple bug fixes to adding functionality we just never got around to, like properly integrating NMM with BOSS. The plan is to talk with would-be developers, discuss what needs to be done (and honestly, you can come to me with ideas for what you think NMM should do and I'll listen), provide rough estimates on how long such functionality would take to code and then provide a respectable financial incentive that both parties agree upon based on how long we think the work will take to complete. I'm not planning to pay a coders hourly wage because the site coffers simply can't afford it (and it would defeat the point of doing this!). Instead, this would be the perfect opportunity for individuals looking for a hobby coding project to earn some money on the side while also getting a rather nice addition to your CV or resumé. If you were looking for a coding job in the web or gaming industry then having Nexus Mods on your CV would definitely help in that regard, and I can, of course, provide references if necessary. Perfect if you've just graduated and you're looking for some names to add to that resumé.

    If this idea interests you and you're looking for a hobby coding project right now then get in contact with me by PM or using the site contact form and we'll have a chat. Neither of us are under any obligation and everything will remain informal, such is my way.

    With all that said, I will simply end by saying that I believe that the worst of the site issues are now behind us. If you've been using the sites over the past week hopefully you've noticed now that things are a lot more stable. However things within NMM still remain quite unstable, and we're looking in to sorting that out as a matter of urgency at this time.
  • 29 December 2013

    End of year network review

    posted by Dark0ne Site News
    At the start of the year I outlined my plans for 2013 with our focus this year on improving the stability and reliability of the service on offer. Now, quite clearly, this hasn't exactly gone to plan. We've hit snag after snag along the way that has caused massive delays, dampened spirits and made for a rather sorry year for us. We're having our most fun when we're coding exciting new features and updates to the sites that you can actually use. We do not have fun working on behind the scenes stuff and it's made worse when it doesn't actually work out, but it really needs to be done.

    We started the year with a hefty sum of money to invest that had been saved up from before Skyrim was launched, primarily from Premium Membership fees. I'd been saving this money over the course of a few years so we could roll out a whole new server setup that would not only provide us with the power we needed but also the quick expandability necessary to deal with the ever increasing traffic and load placed on the servers. This was most definitely the right course of action and I have no regrets on that decision at all. Look at it like you would the gears on your car. In second gear you can only reach a certain speed before you start redlining, you can't go any faster without switching to another gear. Problem is, when you've only got a 2-speed gearbox with no potential to go to third gear the only thing you can do is rip out the old one and install a new 7-speed gearbox. And that's what we did. We installed our new gear box, switched into third gear and opened the way to fourth, fifth, sixth and seven gears in case we need it in the future. What we didn't take into account was just how hard it would be to switch gears and how many unforeseen circumstances you can actually run in to.

    We ordered the database cluster servers at the beginning of February and received them at the beginning of March. We initially thought it would only be a matter of weeks to get things all set up and have the sites moved over to the new database cluster. 3 month's later, at the end of May, we thought we had it but after a straight 48 hours of work we couldn't get it to work properly and had to accept defeat. At which point we paid a rather hefty sum of money for two separate professional consultants to come in and take a look at the setup. It wasn't until the end of July that we finally managed to get our database cluster setup completed and moved to, 5 months after we'd originally started working on a project we thought would take 2-3 weeks. In September I proceeded to buy a further 2 cluster servers, taking the setup to 5 servers, each running 96GB of RAM and dual processors for a combination of 480GB of RAM and 80 CPU cores running at 2.1Ghz (or a hypothetical 160 CPU cores with threading).

    The move to our database cluster helped to remove one issue only to highlight another major issue that needed to be rectified. While we could now serve the database requests we were struggling with a bottleneck with the HTTP requests. Thankfully we'd already earmarked funds for a move to a Cloud setup that would form the basis of our expandability in to the future. Towards the end of July I was in discussions with our server provider to have a Cloud setup specially requisitioned for us. They'd need to build it from the ground up for us so we weren't actually given the keys until the middle of October. All the while the sites on the old setup were on their last legs. Well, they weren't really standing up so much as they were spluttering blood all over the place with multiple puncture wounds in this analogy. Once we got the keys we needed to run extensive tests and mock runs and get it ready for an eventual move over, which didn't occur until the start of December to coincide with our centralisation of the sites. By centralising the sites we managed to make our future jobs a lot easier. No longer were we dealing with 20 different databases for 20 different sites on multiple separate servers, we were now dealing with 1 database with 1 site on a distributed, but for all intents and purposes centralised, server setup. This simplifies things drastically with the down-side being that the problems that were originally localized mainly to just Skyrim Nexus were now problems for all the sites,

    Traffic is at an all time high, as usual...

    If we were still on the old setup then you wouldn't be reading this right now, even if it did take you 5 page reloads and 20 minutes to reach this page. The old setup would simply have been incapable of handling this Christmas/Steam sale traffic. So you can picture this situation like a dramatic Indiana Jones scene if you wish, there's Harrison Ford (the Nexus sites in this analogy), stood on a crumbling platform about to collapse at any second. With a crack of his whip he hooks on to a low dangling tree branch and swings on to a new platform. Albeit this platform still doesn't seem too stable but it's a darn sight better than the previous platform he was stood on which has now fallen into a seemingly bottomless chasm. And this is where we find ourselves at the end of 2013, stood on a precarious platform right now, but the right platform, from which we need to build from and solidify our position.

    The situation is extremely infuriating for us. I've said it before and I'll say it again, we know it's frustrating for you when you can't download your mods or update your file pages but my god, try working 18-20 hour days trying to sort it out in the background all the while trying to answer people's ferocious questions and trying to remain calm. I can't do it and I blow my lid sometimes. It is infuriating to have spent this much money and time on an issue that still isn't resolved. But it will be. It's not a matter of if, it's a matter of when.

    As users you get understandably upset. Lets do a quick Q&A on the regularly asked questions on this topic:

    Q: Isn't it about time you spent more money to fix this issue?

    We've spent close to £150,000 ($250,000) on sorting out our stability issues this year. Aside from the fact it's no longer about the amount of money that needs to be spent but what we need to do with what's already been bought, do you have a spare extra £150,000 lying around that we can use? Please? No? Didn't think so (if you do, by all means get in contact. I can offer you a bag of Nik Naks and half of a Panettone as payment).

    Q: It was way better before. Why not just go back?

    If you're talking from a Skyrim perspective: It wasn't. Your memory has failed you.

    If you're talking from any other Nexus site: it probably was, although it wouldn't be right now in the midst of all the Christmas traffic. You've been lumped in with the same lot as Skyrim now. Your fate is linked to Skyrim's as much as Skyrim's is to yours. It's a necessary temporary down-side in the interest of future stability.

    Q: Why do you keep performing "maintenance" at peak times?

    We put the sites into "maintenance mode" to give the servers time to catch their breath and recover. If we put the sites into maintenance mode they'll recover 90% of the time within 1 minute. If we don't then the problem will remain the same, or get worse, for hours.

    Q: Why don't you work on fixing the issues rather than releasing new NMM versions/updating the sites/eating/sleeping/leaving the house?

    I think by the 20,000+ words written on this very topic this year alone, and indeed the regular maintenance updates and redirects, you can infact see that work is being done. However of the 5 people (including myself) working on the sites only 2 of us can help in this field.

    Telling us to not release an NMM update until the site stability issues are rectified would be like telling a UI designer at Microsoft to not release any fixes or updates to the Windows UI until they've sorted out all their security bugs. The UI designer doesn't work on that aspect of Windows, he doesn't know anything about it and he can't be roped in to help. Telling him to stop working until the bugs on a completely different aspect of the program are sorted would be dumb. And a waste of money.

    I'd like to have another full-time person on board to be able to handle this side of things but there's a few barriers in this regard. Money. The time it would take to hire the person. The time it would take to bring that person up to speed. All negatives for issues we need resolved right now, and not in 3-6 months time.

    Q: I haven't been able to login to NMM since you updated the sites at the beginning of December, what gives?

    So you found the forums and the "new topic" or "reply" buttons but you didn't notice the multiple news posts and indeed the 1,000 other threads and posts about this issue raised in the past week? Sometimes you really need to help yourself. Or do a search! You need to update to version 0.46.0 of NMM. Download it from the site and simply install it over your current NMM installation, making sure your folder locations are the same as your old NMM install. If you do this you won't lose any of your mods.

    Having said that we are going to make documenting things in NMM a bit better over the next year.

    So what has 2014 got in store? Well hopefully we can resolve these issues once and for all very soon and get back to doing what we enjoy doing. I don't think I need to say anything more on that topic.

    At the start of the new year we're going to be decommissioning all our current file servers as they're close to their space capacity, replacing the current 19 file servers: 15 for everyone and 4 for Premium Members with 23 brand new ones, 20 for everyone and 3 for Premium Members. Why a drop for Premium Members? A couple of reasons. We've bought Premium servers in various locations across the globe to act as alternatives to the normal servers. These haven't worked out as well as I'd hoped and they're often unreliable for people. I'll be replacing them with some top quality stuff in Dallas, Washington and Amsterdam that will cost twice as much. I want Premium Members to realise that the real bonus on the downloading front isn't the Premium-only servers but instead is the speed cap being removed and the ability to multi-thread your downloads in NMM. Some Premium Members come undone because they will only use the Premium-only servers when the normal servers are equally or often times better suited for their connection.

    We haven't touched the Image Share section in quite some time. I want to overhaul many aspects of the Image Share section and implement some updated features like galleries, and removing those horrible image pages some users make that scroll on and on into infinity as the author has added their entire back catalogue in to the image description. We'll replace it with a more suitable system that hopefully won't diminish what some people try to do with their descriptions but instead simply make it a lot more aesthetically pleasing and manageable for users. I'll begin consulting with certain users from the Image Share soon, likely to be ones that cause as little drama as possible. No need to get in contact with me, I'll get in contact with you if I'm interested. Image Share drama is the worst drama on the Nexus and I try to steer well clear of it as much as possible.

    I also want to begin work on a Videos section, much like the Image Share section but for YouTubers to showcase their work on the Nexus. I've no interest in hosting people's videos so this system will link straight into YouTube's API system. I think it'd be great to have all the top Skyrim mod video authors, for example, having their own channels on the Nexus from which they can easily showcase their work and the work of the mod authors and you can easily keep up to date with all of them. It won't take away from their subscriber base, it will simply augment it, and best of all for us it won't cost us any valuable bandwidth or server resources.

    On the NMM front we're still working on the 0.50 update with it's profiling features. We want to get it right, and we also want to create a full backup system for users so that they can revert to a legacy system if needed. We've managed to gather valuable feedback from the people using the 0.50 alpha but what we cannot ascertain is how successful it's been for users as a percentage. We don't know how many people it hasn't worked for, e.g. how many people couldn't update to 0.50. We want to create a version of NMM that we'll likely dub NMM Legacy. This will be a version of NMM that we shall feature-freeze at 0.46.0 for ever but that we'll always ensure is able to access the NMM web services (for logging in/downloading/mod version checking). It's our hope that each time we release a major update to NMM we'll ask you if you want to backup your current copy of NMM. If the update fails for any reason then your backup will work with NMM Legacy, so you won't have lost anything.

    We've also got that NMM design overhaul to look in to. I'll be writing up a blog post about NMM soon as it seems some people are upset we're still in Beta after a couple of years. I'll set the record straight on that one because we're not coming out of Beta any time soon.

    I've absolutely no idea how long the stabilization of our services is going to take. I'm not even going to fathom a guess because each time I do, each time I assume we're close, we hit another snag. We've now got a huge backlog of stuff to do after the centralisation, however, so I hope we can get some updates out thick and fast soon.

    I hope your Christmas, New Years and indeed your entire year has not been as stressful (or expensive) as ours has. I hope that 2014 brings us all more success.