MORROWIND

View:

  • 21 March 2013

    NMM bug fix release and download hacking problems fixed

    posted by Dark0ne Site News
    We’ve released a bug fix edition of NMM, taking us to version 0.44.4. Here’s the change log:

    1. New Feature: The ReadMeManager now supports PDF files.
    2. Bugfix: ReadMe Manager preventing .txt files from being installed in mod folders. (This fixes the installation of mods like Fores New Idles)
    3. Bugfix: Rare crash while deleting a mod in CategoryView.
    4. Bugfix: Omod script interpreter.
    5. Bugfix: Users were able to set Mods and Install Info as the same folder.
    6. Bugfix: NMM crashing with a corrupt ReadMeManager.xml file.
    7. Bugfix: NMM using the wrong game path after a game rescan.
    8. Bugfix: LoadOrder export using the wrong plugin list.

    You can download the update through NMM or by downloading and installing manually through the NMM download page.

    Unfortunately earlier today we found out that some of our file servers had been hacked, replacing manual (not NMM) downloads with a malicious installer that contained malware. This hack was targeted as the hacker deliberately named the file “Nexus_Downloader.exe” to try and snag as many people as possible. While I haven’t tried to run it myself it seems to be a scam malware that sends the user to a fake FBI page informing the user they must pay must in order to unlock their system. To make it worse only 4 of the 20 main anti-virus programs people use flagged this as a virus. It’s obviously a relatively new one that most haven’t caught up with, and this might have caught a few of you off guard.

    As soon as we found out about the breach we had the servers down and patched up within minutes and we’ve been working today further hardening our servers and shutting down the method used by the hacker to gain access.

    There’s a few things to take away from this. First of all I’m sorry that this has happened to some of you. You clearly trust us to provide you with a safe and secure modding experience and we got caught with our pants on our heads. While this won’t help you now, can I give you some advice? Don’t trust anything, any site, or any person fully on the internet. Be suspicious of everything. If you’ve tried to download a fluffy sheep mod that says it’s 50mb in size for Skyrim and instead been given a file called “Nexus_Downloader.exe” that’s 100kb in size...be suspicious. We’d never do something like that, especially without letting you know first. These sorts of things can happen to any site out there (just look at the past hacking’s of Sony, Valve/Steam...even Bethesda got hacked) and having an anti-virus, firewall and malware protection won’t keep you 100% safe. Nothing will. But try and be perceptive and don’t let your guard down.

    We can’t guarantee your safety 100% when browsing and downloading from the Nexus. No other site can offer you that guarantee either. What I can guarantee you is that we work hard to try and make the experience as safe and secure as possible, and we work even harder when we know we’ve been breached, often without sleep.

    Never be afraid to report suspicious activity either by others or by the sites themselves on the forums or to the staff.
  • 18 March 2013

    Recent site updates and NMM 0.44.3 released

    posted by Dark0ne Site News
    We’ve been knocking out tiny updates and tweaks to the site since we changed the design around a bit a week or so ago. These updates tend to be small enough to not warrant a news post, but when they build up enough I like to smash them all out in a news post in one big list. But before we do that let's get NMM out of the way.

    We released NMM 0.44.3 last Friday that contains some more bug fixes and some minor feature changes. Here’s the list:

    1. New Feature: NMM will now prevent ReadMe files from being extracted in the mod folder (thus only installing them in the ReadMe Manager), you can set this behaviour on/off in the Settings menu (Default on).
    2. New Feature: The user can now select the game mode by double-clicking on it.
    3. Bugfix: Mods failing to deactivate on upgrade.
    4. Bugfix: Category view failing to update after mod rename.
    5. Bugfix: Mod list not re-sorting after mod/category name change.
    6. Bugfix: NMM mods startup scan searching in the readme folder (causing some duplicate entries in the mod list).
    7. Bugfix: NMM crashing when another instance of the program is already running.
    8. Bugfix: "Yes to All" not working as intended.
    9. Bugfix: Some OMOD crashes (comment lines and "misplaced" curly brackets).

    We’ll continue to drill out some bug fixes in coming updates, and the next major feature we’re working towards is mod profiles; the ability to have different modding profiles for your games so you can have different mods installed and running in your games for different saves/instances. We don’t want to introduce a DLL hack to do this, though, so we’re exploring other ways of doing this with NMM so we can get profiling in every game NMM supports. We then hope to further expand on that functionality so you can share these profile setups with others on the Nexus sites, like Steam Workshop Collections, but done slightly differently. That’s the plan, anyway.

    As always you can get the latest version of NMM from the download page

    And now the site updates:

    1. The arrow next to your account information in the top right corner is now clickable so as to be less confusing.
    2. Changed the search button so it highlights the textfield, bringing the search functionality back to a one-click process.
    3. Added the ability for advanced search options in the filename field, including using quotes to group words, the “OR” operator and the “-” operator. Instructions on this functionality can be found on this wiki page.
    4. If you have tracked a file the wording will change from “Track” to “Un-track”. If you have endorsed a file the wording will change from “Endorse” to “Unendorse”.
    5. Fixed an issue with the vote, track and endorse buttons being a bit temperamental and sometimes not working.
    6. If you have not voted for a file this month the vote button will be green. If you have voted for the file you are looking at the button will be orange. If you have voted for a mod, but not the mod you’re currently looking at, the vote button will be yellow.
    7. Added a confirmation popup for voting to let you know you’ve already voted for a file this month, that you can only vote for one file in a month, and if you vote for the current file you’re looking at then your previous vote will be removed.
    8. Adult-only files are now eligible for the hot files, provided the user has turned adult-only content viewing options on.
    9. Added a “reply” link to every reply to a comment so that you don’t have to scroll up to the original comment in order to reply (replying to a reply is just treated as a reply to the original comment, however, and there’s no added nesting).
    10. Replying to a comment will now bump the comment (and replies) to the top of the comments. If you don’t like this functionality you can turn it off in your preferences under “Replies to posts bump the original post”.
    11. We have reduced the wait time between downloading a mod and endorsing a mod from 3 hours to 15 minutes.
    12. The “Download with NMM” button no longer shows in the file header if a file has more than 1 main file. The user will have to go to the “files” tab and choose which file they want to download.
  • 07 March 2013

    Site design update, file backgrounds and hot file cropping

    posted by Dark0ne Site News
    If you’re reading this from a Nexus site then you’re reading it from our updated design. Before I go into explaining a few of the changes just a quick note that we’ve changed our imagemap, so if some images look out of place or things look like they’re in the wrong place you might want to refresh your browser cache for the site. Easiest way to do this is to press CTRL+F5 (a hard refresh). If that doesn’t fix it then try again. And again. If it’s still broken after that it’s either us, or not “broken” and you just think it looks wrong.

    As promised I’ve made some changes to the layout that aren’t too different from the original. So, far from being an overhaul, it’s more an update to the original design. Right now we’ve made changes to the headers of all the Nexus sites and to the file page layout. We’ve got a few more page layout changes in the works, and next will be the category/search result page, but right now lets run through the current changes.

    As mentioned the header has changed. The search, messages, notifications and account options are all accessible from the top bar now and we’ve removed the need for the originally (relatively garish) coloured bar, thus saving a few pixels in height in the process. Clicking the search button will produce a bubble box for you to do a quick search, and we’ve also made it more obvious that we have an advanced search as well. One thing that really surprises me from doing the internet rounds is that lots of people still don’t know we have an advanced search feature to really drill-down into what you’re looking for.

    We’ve changed our logo area a bit; each site is labelled as Nexus Mods and the name of the game is listed below it. This is going to fit in to our future plans a bit more, so this change is more a change for the future rather than a necessary change right now. The normal navigation is still in-tact from the original design, no changes there, so you shouldn’t get lost.

    The file page changes are slightly more substantial.

    Probably the most notable change we’ve made is adding a file header background image that mod authors can create themselves through the site interface, to bring some more colour and variance from other mod pages out there. Something that makes your page look more unique from others. As is our way, creating a file header for your pages is completely optional, so if you don’t like them on your file pages you don’t need to do anything, you can stick with the default look (or you can play around with the system, then delete it if you don’t like the look).

    To create a file header you need to go to your file images and click the background image icon underneath the image you’d like to use. You’ll be taken to a page where you can crop the image you’ve selected (much like you might crop your profile image on a social networking site like Facebook) and see how it’ll look on your page before you decide on whether you want to use it or not.

    In the same vein we’ve also produced a similar tool that allows you to create your own hot file images for the site to use. Up until now the site has automatically generated these images and it’s been a bit hit and miss. Sometimes it produces a great hot files image, other times it produces an image of..the sky...or the sea...or a head...and it’s not so great. Now it’s all on you. If you think this feature is redundant for you because all your files were released a while ago and are no longer eligible for the hot files system then think again. We’re bringing in a “Feature file” system for the new category page. This system will show a random file from the category you’re viewing that changes every 10 minutes. Every file with a hot file image is eligible for the Featured File system, and we’ll rotate properly so every file gets equal time and exposure. We’ll also let you know how many times your file has been featured on a category, just for your own peace of mind. So creating a hot file image on your old files is highly recommended if you’d like some added exposure in the not too distant future.

    We’ve moved the buttons to download the file, endorse it, vote for it and donate to the author to the top right in prominent green colours. We’ve also added a big “Track” button as well, as lots of people didn’t know we had a file tracking system. Clicking the vote, endorse and track buttons will make them turn orange (if it was successful), so you know when your vote, endorsement or track action has worked. They’ll remain orange when you revisit file pages, so you know at an easy glance whether you’ve already endorsed or tracked a file or not.

    We’ve added some text below the “Action” buttons which should make it more obvious what each one does without being a mystery meat navigation that you can’t intuitively work out without hovering over the tiny icons. Similarly we’ve increased the size of the file tabs themselves and added a description at the top of each tab, once again to make it more obvious what each tab is for.

    And that’s it!

    If you think something isn’t looking right or working properly then feel free to let me know. I’ve tested the site in the latest versions of IE, Fire Fox and Chrome, but can’t account for every variation.
  • 06 March 2013

    NMM version 0.44 released, new file servers and site updates

    posted by Dark0ne Site News
    We’ve snuck the latest version of the Nexus Mod Manager out the door late this evening pushing us into version 0.44. This latest version contains a number of bug fixes along with a new ReadMe management feature and the new improved controls for the “old view” NMM mode, the same that the category view uses.

    The ReadMe manager parses all your mods for text, rtf, doc and HTML documents and provides them as a list for you to open straight from NMM by right-clicking a mod and highlighting the “Open ReadMe file” element, making it quick and easy to view the ReadMe’s for your mods. Unfortunately this feature highlights a worrying trend for me; a lot of the mods I have installed do not have a readme included in the archive with the mod. This is bad form, mod authors. You should always try and include a readme with your mods containing at least a description of the mod, the author name, any credits, your license information and, if you’re looking for feedback, a preferred contact method (whether it’s a link to a forum thread, your email address, a username on a forum people can PM, etc.). Any sort of identifying ReadMe is better than none.

    Now that the non-category mode for NMM has the updated control system you can do some of the cool stuff that originally only the category-view could, like clicking the columns and sorting them by ascending and descending. Why is this handy? One example would be when you install a new mod and then can’t find it. Go into non-category view and sort the column by install date descending. Now you can see your mod list based on when you installed your mods.

    A number of bug fixes have also been implemented into this build. You can find the full changelog here.

    On the site front we’ve added a number of bug fixes for the sticky posts and nested commenting. You can now properly edit your replies without seeing the additional HTML that makes the forum posts a bit more understandable as replies. Mod authors replying to comments in their own mod threads now have the proper coloured borders that make their posts more noticeable and prominent. Replies are now ordered by date ascending to make a more logical and readable thread of replies and mod authors can now lock their sticky posts so they don’t get spammed by off-topic replies.

    We’ve added an additional 2 servers in the Salt Lake City area, bringing our total for file servers up to 12 plus 2 premium file servers.

    We’re very close to releasing an update to the Nexus site design which will begin with an update to the site header and file pages. Mod authors, you can find a mockup of the new page layout in this private thread. These aren’t massive, extensive visual changes. Much like you can still tell Microsoft Word is Microsoft Word whether you’re using the 2000, XP, 2007 or later versions, you’ll still recognise the sites.

    Lastly we’ve changed the Supporter membership, as promised, so that Supporters no longer see ads on the site. I’ve been working hard looking into alternative payment platforms besides Pay Pal for those people who have a chip on their shoulder (or who can’t actually use Pay Pal due to locational difficulties) but after lots of extensive searching I can tell you one thing; there’s a reason why Pay Pal are so dominant. They make it so easy; they don’t expect you to jump through ridiculous bureaucratic hoops just to be accepted or have ridiculous rules and their system integrates perfectly with almost everything out there, including our system. While some alternatives are out of reach simply because they’re not open to EU residents yet (Stripe) unfortunately some have ridiculous stipulations. Take for example Google Checkout, who have stipulated in their rules that you must use their payment button to direct people to their payment process. Our system isn’t designed to work with custom buttons so right now, we can’t use Google Checkout/Wallet. Pants on head stupid rules that serve absolutely no purpose. We’re looking at several payment gateways, but all of them will require extensive time to make them work with our system, which is time spent away from actually working on the sites. It would be a lot easier if you people would just used Pay Pal (if you can), but hell, I can’t force you!

    I’ll be sure to keep you updated on the payment gateway front if and or when I have any further news.
  • 24 February 2013

    OpenMW v0.21.0 released!

    posted by OpenMW Game News
    The OpenMW team is proud to announce the release of version 0.21.0! Release packages for Ubuntu are now available via our Launchpad PPA. Release packages for other platforms are available on our Download page. This release introduces video playback, improvements to cell load time, and parsing for escape sequences in dialogue and message boxes.

    Check out WeirdSexy's release video.

    Known Issues:
    • No sound when playing videos on OS X
    • Extreme shaking may occur during cell transitions for some users (enable anti-aliasing as a possible workaround)
    • Launcher crash on OS X < 10.8


    Changelog:
    • Various dialogue, trading, and disposition fixes and improvements
    • Torch flickering improved to better match vanilla Morrowind
    • Fix for attribute fluctuation when infected with Ash Woe Blight
    • Adjusted activation range to better match vanilla Morrowind
    • Fixes for the Journal UI
    • Fixed crash caused by Golden Saint models
    • Fix for beast races being able to wear shoes
    • Fix for background music not playing
    • Fix for meshes without certain node names not being loaded
    • Fix for incorrect terrain shape on inital cell load
    • Fix for MWGui::InventoryWindow creating a duplicate player actor at the origin
    • Added video playback
    • Added support for escape sequences in message box and dialogue text
    • Added AI related script functions, note that AI is not functional yet
    • Implemented fallbacks for necessary ini values in the importer, unused in OpenMW as of yet
    • Implemented execution of scripts of objects in containers/inventories in active cells
    • Cell loading performance improvements
    • Removed broken GMST contamination fixing mechanism


    see also:
    *Official Youtube Channel*
    *For New Developers*
    *OpenMW Homepage*
    *Chat @freenode*

  • 22 February 2013

    Recent updates: nested commenting, sticky posts, latest versions, drag n dropping and bug fixes

    posted by Dark0ne Site News
    Things have been flying by a bit recently and we’ve been boshing out so many small fixes, tweaks and features left right and centre I’m almost struggling to keep up. So here’s a run-down of some of the recent changes we’ve implemented on the sites and in NMM.


    Nested commenting

    I’ve been wanting to get this functionality implemented on the sites for a long while now but there was no point getting it done until we upgraded the forums (which was done within the last fortnight). Nested commenting allows users to reply to comments made on the sites and have those comments and replies grouped together for easy viewing.

    Before nested commenting everyone’s comments were shown in a linear progression, one above the other, ordered by the date and time the comment was posted. It meant if you asked a question on a file page at 10am and the author (or anyone) couldn’t answer that question until 5pm, there could have been any number of comments (sometimes even hundreds) made on the file page in between the time you posted and the time a reply could be posted. That made finding the answer to your question very hard. Now, you can reply to the comments people make, and because they’re grouped together you’ll easily be able to see if anyone has replied to the comments you’ve made.

    We’ve only provided one level of replying to comments. So comments only get nested (and indented) once. You can’t reply to a reply and have that reply indented further. Hopefully that makes sense. So you can’t have these crazy nested reply chains like you can on a site like reddit. It gets a bit messy!


    Author’s sticky post

    Something I know authors need help with is increasing awareness to their users about various things from incompatibilities, common issues, install instructions and what not. The inherent problem with the internet is there’s a percentage of users who seem to suffer from a form of attention deficit disorder and can’t spend the time reading the file description, readmes, FAQs and author warnings in big glaring multi-coloured fonts before they try to install a mod. If they can’t get it to work they then go to the comments section, completely by-passing everything the author has written on the topic in various locations and start asking for help. That’s REALLY frustrating if you’re a mod author. Unfortunately there’s not much you or we can do about that. If someone is really that dumb that they don’t read anything before asking for help then no amount of warnings, pop-up notifications and forced reading will solve the issue (much like our terms of service agreement boxes, or the big glaring boxes in the image share telling people not to upload nude images).

    However, we’ll try to help with this issue in whatever way we can, and to that end we’ve introduced a “sticky post” system that allows file authors to sticky a single post in their comment thread. The post goes to the top of the comment section, where it is “stuck” there on every comment page, with a red border and a notification that it’s a sticky post from the author.

    As an author this has some practical uses. Are you aware there’s a bug in your latest version that you’re working hard to fix? Maybe some major incompatibilities? People keep on bringing up the same issues that aren’t actually issues with your mod? Sticky a post to the top of your comments to let users know they don’t need to spam your comment section with bug reports about it. I’m sure you’ll still get bug reports from those previously mentioned users who DO NOT READ before they act (RAGE!) but hopefully you’ll be able to catch a few more people with such a notification that should help to reduce the load.

    Something I’ve noticed from my time using the PC Gamer website, where they have nested commenting, is that the comment at the top of an article can often be used by people to talk about things completely unrelated to the original post. While we haven’t coded in a system to “lock” your sticky post (so people can’t reply to it with unrelated stuff) if it becomes obvious that it’s needed, we’ll do it. I’ll await mod author feedback for that one.


    Latest version check box

    The versioning system on the Nexus is far from complex, uniform or standardised. We let you guys version your mods however you like. Unfortunately sometimes this can lead to inconsistencies, especially with NMM where the latest version column checks the file version the user has installed against the file version shown on the file page (via the “edit attributes” page), not the file versions of your individually uploaded files. So if your file page says the latest version is v1.2 and your uploaded files have a file that’s v1.3, your latest file version is still v1.2.

    While we want to visit the versioning system in more detail later, especially in regards to NMM, right now we’ve just added a quick and easy system for you to update your latest versions. We’ve added a checkbox when uploading your files that asks you if this is the latest version of your file. If you tick the box the latest file version information for your file will be updated with the content of the “file version” text field you’ve used in the upload form. Saves you going back and editing your attributes with your new version.


    NMM drag n drop
    Version 0.43.1 of NMM has added a small but useful feature for those people who still download their mods manually from the sites. You can now drag and drop files from your desktop into NMM directly, avoiding having to use the “add from file” functionality.

    Right now this feature only works while in category mode. We’re working on updating the non-category view to use the same controller as the category view so it can make use of all the cool features the category view has, like being able to sort the columns, and the drag and drop functionality.


    NMM bug fixes

    Along with this feature update the latest version of NMM also contains some bug fixes. Here’s the change log:

    *InstallLog didn't properly check for already present random mod keys (this will prevent the "An item with the same key has already been added" crash).
    *Mod autoupdate check removing category info from mods, causing missing mods from categories.
    *Uac check crashing with illegal folder paths.
    *Scripted mod install failing when the installer tried to show a message to the user (throwing a Security exception).
    *ProgressDialog bug causing mod update checks to fail.
    *Illegal folder paths crashing the game scan process.
    *The "move to category" ContextMenu didn't properly sort alphabetically.
    *Prevents the user from settings Mods or Install Info folders equal to: HD root, game root, installed mods folder.


    What we’re now working on

    Moving on we’ve got lots of new features in the pipeline for both the sites and NMM.

    Right now I’m working on some updates to the site’s design, which I’m sure will fill a lot of you with fear and dread. It’s not going to be a full redesign of the sites like we had last year at all. You’ll still recognise the sites, I promise. If you’re a mod author you can see my mock up for the file page redesign in the private mod author forums that I’ve appropriately titled “fixing things that aren’t broken”. I hope to do the changes progressively. So I’ll be changing the header about a bit to better manage the space. We’ll do that first. Then we might update the file pages. Then the category view. Then the site index. I don’t have plans for any other pages just yet.

    The clever site coders are working on the bandwidth throttling system based on ads and endorsements that I brought up in my last blog post. They’re also working on a new feature for mod authors that’s going to let them pick and crop the images they want to use for their hot files and “feature image” on the new file page.

    Our hot file system is quite unforgiving when it comes to picking and cropping the images it wants to use, and how it crops them, so we want a system where you can pick the image the site will use as it’s hot file image (if your file becomes a hot file) and also pick which area of the image gets cropped so it advertises your mod in its best light. This will be handy not just for people with popular mods because I also plan to use hot file images in the new category design, where we’re going to have a “featured” mod; a mod that is randomly picked from the category that will be advertised in the category for a set amount of time, and then sent to the back of the pile before it can be shown again as a feature mod. A form of banner advertising and rotation for mods, if you will.

    On the NMM front we’re working on the aforementioned update to the non-category view, more bug fixing, better readme management and our initial work into creating a really easy to use mod packager for authors who want to ensure their mods work with NMM and, drum-roll, better scripting support as a part of it. We want to make it so that mod authors who don’t know how to script and don’t want to learn can still make some pretty neat installers that give their users more options when installing.

    We’re aware there’s still some niggling issues with the forum upgrade for those people who use the forums. We’re aware of them (especially the VERY annoying bug that is causing edited posts to come up in garbled HTML at random), but some things are taking a higher priority right now. Word of advice that applies at any time; always copy and paste your long posts into notepad before posting a response.
  • 11 February 2013

    Forum upgrade complete, still some work to do

    posted by Dark0ne Game News
    It’s been a long day but 8 hours after taking the forums offline we’ve now put them back online with the necessary updates in place. There’s still quite a bit of work to do, tweaks and performance enhancements to make, but the majority of the functionality should now be in place and we’ve re-enabled mod publishing and commenting on the sites.

    We’ve all got to get used to the changes that have been made by this latest version and we’re still trying to find our way around things and remember all the old settings we had in place before. While we use Invision Board for our forums we’ve made a hell of a lot of edits and tweaks to both the software and the servers over the years to be extremely optimised for the amount of traffic the sites receive. This latest update has wiped out all of those optimisations so we’ve got to try and get it back to how it all was before.

    We’ve been testing out the new software over the past couple of weeks and while we can test for most things, we can’t stress test for the 100’s of requests the forums receive every second when things go live. Please bear with us, everything should be as it should be soon.

    In the meantime if you’re one of our forum users feel free to do some exploring. I’m sure a lot has changed in the software. Don’t like how the posting forms now look? Me either.
  • 09 February 2013

    Scheduled partial down-time and mod publishing

    posted by Dark0ne Site News
    Something we rolled out on the sites a couple of weeks ago without any kind of announcement was a new file publishing system when adding file pages to the Nexus. The inherent problem is thus; if you want to (properly) release a mod on the Nexus you need to first start a file page, add a good long description, add and setup all your images, upload your files and setup any videos and mirror links. If you do this without hiding the file (so that others can see it) you run the risk of some hasty (and annoying) users asking you why you haven’t uploaded your files/images yet. If you do it while your file is hidden then your file is losing “valuable” exposure in the new today and recent files pages and columns.

    So what we’ve done is added an extra step to the process. You no longer choose whether you want your new file page to be visible or hidden. All file pages start off hidden and you now need to “publish” your mod before it becomes visible on the site. Until you publish your mod your file page will have no comment thread, no upload date and you have all the time in the world to upload and setup everything exactly how you want it before your mod is visible to everyone. It also guarantees you complete control over when the file is released to the public, and as the upload date isn’t updated until you publish the file, you can have your file unpublished for as long as you like without suffering any drawbacks.

    For an added example of why this might be handy, take if you will a new mod you’d like to upload to Skyrim Nexus. You think this mod is going to be the next big thing. Now the Hot File system on Skyrim Nexus has a 7 day turn around time, so it only shows the most endorsed files uploaded over the past 7 days. If your mod is 1GB+ in size it might take you a very long time to upload it to the site, anything from a few hours to days depending on your upload speed. So by the time your file is actually up on the site and ready to be endorsed you’ve lost a hell of a lot of time, and perhaps dented your chances at hitting the (sometimes) coveted hot files. With the publishing system in place, your mod only gets properly added to the site when you click that publish button, ensuring you’re absolutely ready to launch. Now, if you don’t care about the hot files or your mod’s exposure then this isn’t a big deal for you, but I know it is for some. So that’s what the publishing system is for, and it’s going to come in handy for the partial down-time I’m about to tell you about.

    We’re hoping that on Monday (the 11th of February) we’re going to be taking the forums down for a long overdue software update. Now the forums are pretty integral to the operation of these sites, and lots of functionality is linked to it; the commenting system, the user login and registration system, messaging, chat, and so on and so forth. We’ve been running some tests over the past couple of weeks, doing some mock runs, and we believe we can keep the majority of the site’s functionality operational during this forum downtime. You’ll still be able to browse the sites, download mods, look at images and even read comment threads, but you won’t be able to browse the forums or post any new posts or comments either on the forums or in file or image comment threads. Similarly new registrations will be locked down during this period.

    As the forums are going to be locked down during the upgrade we’re also going to lock down the publishing of new mods to the database, as publishing a mod during the forum upgrade would result in the mod comment topic being lost. As such, you will be able to add or edit new file pages, upload files, images and everything else you do before publishing a mod, but you won’t be able to actually publish the mod so it’s visible to the public until the forum upgrade is finished. This only affects new file additions to the site. You’ll be able to manage any current files already published on the site no problem. We’re doing this for the integrity of our data, and for the safety of maintaining the effort you’ve gone into getting your file up on the site.

    We’ll be putting notifications up on the site to let you know when we’ve taken the forums down and I’ll try to keep you updated. This is one of these things where any number of problems can occur along the way. We’ve tried to account for everything (and there’s a hell of a lot to account for considering how integral the forums are to the running of the sites) but it’s logical to assume that at some point something unexpected is going to happen that we’ll have to react to. So while I’d like to tell you that this forum downtime will probably only last 6 hours, it could last 12 hours, a day, or hell, even longer. But I hope not.

    Fingers crossed!
  • 05 February 2013

    Blog piece: Ads, Supporters, Endorsements and Bandwidth Throttling

    posted by Dark0ne Site News
    I want to use this article to talk about the ads on the Nexus network, the introduction of a new ad format and the choices we’re going to offer you in regards to these ads. We’re making Supporter membership an ad-free experience for the one-off cost of £1 and I’ll also be delving into how we’re planning to incentivise the endorsement system a little to try and increase the download-to-endorsement ratio for mod authors. First let me explain our bandwidth throttling system, which is going to be used to incentivise aspects of the the site explained later on in the article.

    Last October when we upgraded the download system we improved our bandwidth throttling capabilities and upped the bandwidth limit on the site for normal members from 500kb/sec to 1MB/sec. The increase is obviously good, because you should be getting faster downloads, though it’s confused some people as the bandwidth limit is across all your files you’re downloading. So if you’re downloading 1 file, the maximum speed you can get is 1MB/sec. If you’re download 4 files the maximum speed you can get across all your files is 1MB/sec, or 250kb/sec per file if it averages out. So the more files you concurrently download, the slower speeds you’ll get across all of them. I wanted this more manageable bandwidth throttling in place so that we could provide incentives for members of this community who help the site, or who are particularly productive and active.

    Something I touched on in my last blog post was the financial side of the Nexus; what I do in regards to money, why I do it and how the sites cope. We’re planning on completely overhauling the server infrastructure of the network this year to massively improve reliability and sustainability into the foreseeable future. The obvious obstacle in such an endeavour is the huge cost, and it’s my job to find ways to not only afford our current overheads but also to save up for this new infrastructure on top of all the current overheads.

    As a network we have huge overheads to pay for; 18 super powerful servers, an average bandwidth consumption of 2.8Gbit/second (which is over 800TB of bandwidth a month), 4 dedicated, full-time programmers for the sites and NMM, software expenses, accounting and so on and so forth. As you can imagine, the costs are extremely high, but we do get by.

    We get by for two reasons; ads, and Premium Members. They’re both ridiculously important to the survival of these sites, and if either source were to dry up then the Nexus would no longer exist. That’s not scaremongering, that’s just a fact. We can’t run these sites completely ad free like, for instance, Steam Workshop can because, unlike Valve, we can’t subsidise our mod distribution service with the ridiculous profits we’re taking from sales of games on Steam. Mods are our primary dedication, not a secondary service we provide as a means to bolster our primary sales business.

    There’s two inherent problems with ads. The first is my lack of focus on direct ad sales, that I touched on in my last blog post, which means that we scrape the barrel in terms of ad revenue. Because we’re not directly selling our inventory to premium advertisers we’re not making mega-bucks off of them like the big gaming powerhouses with their huge ad sales teams. The second is the prevalence of ad blockers on the internet. Software and plugins like AdBlock and NoScript are extremely simple to install programs that eradicate almost all the adverts on the internet. And that’s great for you, because no one likes seeing ads. But it’s not great for webmasters out there who rely on advertising to afford the upkeep on the sites.

    There’s various different reasons why people use adblockers. We all hate ads, and if you have a choice then you’d obviously choose to not see ads over seeing ads. Some people worry about security, some people just don’t care about the sites they’re visiting and just want an ad free experience, and others seem to understand that websites need to make money to afford the upkeep but think that their adblocking doesn’t affect revenue as they never look at/click on the ads when browsing anyway. And that’s where you’d be wrong. To clarify the point, I’ll quickly explain the three main advertising revenue models; CPA, CPC and CPM:

    • The Cost Per Action model pays out money for a specific action a user might perform after clicking an ad. Say it’s an ad to sign up for a second hand car newsletter. If you click the ad and add your email address to the newsletter, there’ll be a payout for the displayer of the ad.
    • The Cost Per Click model pays out money when a user actually clicks on an advert on a site. Pretty self explanatory.
    • The Cost Per Mille models pays out money per one thousand (mille) impressions shown. You don’t need to interact with the advert to be paid, you don’t even need to click on it. If that ad gets shown to the user, you’ll get paid. And that’s what we use.


    So as we use the CPM model of advertising, you can see that even if you never look at an ad and certainly never click on one, just by that ad being shown you’re helping to support the site through ads. The same cannot be said if you use an adblocker because the ad never gets shown.

    In this day and age I have a lot of respect for those people who could install an adblocker but don’t, or for those people who have an adblocker installed but choose to disable it on the sites they like and want to support. Even if you can’t afford a Premium Membership, If you do this, you’re supporting the Nexus financially and helping to pay for the upkeep of the sites. Without you, we couldn’t afford to keep these sites running. And I respect that.

    It’s at this point you hit this sort of moral dilemma. You’re a webmaster running a site that absolutely depends on the advertising revenue from the sites in order to keep them up and going. There are people out there either not using adblockers or who are using adblockers but have them disabled when browsing the site. These people get a lesser browsing experience with ads, but they’re helping to support the upkeep of these sites. And then there are people out there who are blocking the ads on this site, and getting a better browsing experience for it, and who contribute nothing to the upkeep of these sites. Is it fair? If you’re an advocate of adblockers you’d probably say yes, it’s fair. As a webmaster who relies on advertising revenue, I’d say no. It’s not fair.

    The inherent problem is thus; if everyone visiting these sites blocked the ads then this site would no longer exist. Once again, that’s not scare or guilt mongering, that’s a fact. I don’t think it’s fair that people who choose to see the ads should get a worse experience browsing these sites than those people who choose not to see the ads but who don’t help to support the sites.

    So what am I going to do? Two things.

    First of all we have two different types of financial incentives on this site, that can be purchased through your account on the forums. Supporters and Premium Membership. Premium Membership provides all sorts of bells and whistles to your Nexus experience like uncapped downloads, multi-threaded downloads, Premium only file servers, full download history and so on. It ranges in cost from £2.49 a month (around $4) to £39.99 for a lifetime membership (around $64). If you buy a timed Premium Membership, let it expire and then don’t renew the payment we move you automatically into the Supporter membergroup. You have supported these sites financially, and you deserve to be recognised even after your payment has expired.

    A Supporter membership on the network costs £1 (around $1.60). It’s a one-off payment that makes your account a Supporter account for life and it works on all Nexus sites, current and future. You get a couple of very minor bonuses like access to the Supporter-only Image Share section and an increase in your Private Message capacity from 100 to 500. We’re now going to make it so that Supporter memberships also provide an ad-free experience on the sites.

    Why make Supporter membership ad-free? It’s very simple really; if everyone who used this site for more than a couple of downloads donated a one-off £1 payment to the upkeep of these sites then not only could we get rid of all the advertisements completely, but we would have around 15 programmers, a server network that would only go down if Skynet attacked, no download speed limits and an all round better experience. So it makes sense that, for the princely sum of £1, you have more than paid your dues to the Nexus and you should be rewarded with a completely ad free experience.

    When I look at the reasons people give for why they use adblockers I often see “ad free membership is too expensive”. I don’t think that argument could be used here now.

    That’s the first thing we’re going to do, and we’ll implement that within the next month. The second thing we’re going to do is drop the download speed limit on the Nexus to 750kb/sec, a 25% drop on the current 1MB/sec download speeds. What we’re then going to do is run some code that will tell us if you’re seeing our ads or not. If you’re seeing our ads, we’re going to provide you with a bonus download speed of 250kb/sec, to take you back up to the original 1MB/sec. You’re helping to financially support these sites, and I honestly believe you deserve a service above those people who do not.

    I want to be careful here and ensure that we don’t cripple the crap out of the service we provide to people using adblockers. If you want to use an adblocker and block the ads on this site then that’s completely your prerogative, and I don’t want to do something silly like say “you can’t download unless you turn it off”. That’s dumb and it causes unnecessary friction between you and the site, and that is not my intention. My intention is to recognise those people who do not block the ads on this site and incentivise the support their showing by having the ads turned on. You use an adblocker out of choice, and now you can decide whether not seeing ads on this site is worth a slower download speed or not. I’m sure most of you will still prefer to not see ads for whatever reasons you might have to use the adblocker in the first place. That’s absolutely fine, that’s completely your choice, and for you, your use of these sites will basically be exactly the same apart from some slightly slower download speeds.

    Once again, that’s something we’ll implement in to the sites over the coming month.

    Before I move on from talking about adblockers, I just want to add a quick note. Some people even go the extra mile and ask me if clicking the ads every once in awhile would help us more. While I thank you very much for even considering the idea my answer is always the same; please only click on an ad if you’re actually interested in what the ad is about. It serves no purpose for you or us if you just click an ad for the sake of clicking it. It artificially inflates our numbers and doesn’t help the advertisers who want to see a return on their investment. So once again, thanks a lot for even thinking about it, but honestly, don’t bother unless you actually like what you see.

    Lastly, on the ad front itself, I want to announce my intention of adding a new ad format to the sites on top of the current ad formats. Site skin ads. You’ve probably seen them around on other sites if you run at resolutions above 1152 pixel widths; they’re ads that make use of the blank spaces either side of the central content area. It is my intention to start selling this space to gaming-related only advertisers in the not too distant future. Now, before you grab your torch and pitchfork I’m going to put your fears to rest. I know site skins can be really annoying. I know, because they annoy me when I visit other sites. So this is going to be optional, and you’re going to be able to turn them off in your preferences. And why would you not want to turn them off? Well, aside from the fact they’re going to help to pay for all the new servers we’re buying, we’re going to incentivise it with the bandwidth throttling again. If you choose to leave the site skins on, we’ll give you a further 250kb/sec bump on your bandwidth, for a total of 1.25MB/sec speeds. Ooooo, aaaaaah. And we won’t penalise you if you turn them off. So if you use an adblocker and block all the ads, you’ll get download speeds of 750kb/sec irrespective of any settings. If you have ads on but block the site skins in your preferences you’ll get 1MB/sec. If you have everything on you’ll get 1.25MB/sec. It’s not exactly much, I know, but the thought of providing (even slightly) better service to those people who grin and bare the pain of ads at least makes me feel better, while helping to support the upkeep of the sites.

    Right, I’m done talking about ads now. I’m already 2,200 words in and I’ve still got to talk about how we’re going to incentivise the endorsement system.

    So the endorsement system. The endorsement system is there to provide mod authors with a feel good factor; a feeling that what they’ve uploaded to the sites is actually being used, and liked, by others. It tells them that they’ve done well. So what’s the problem? The problem is the very low download to endorsement ratios currently on the site.

    If we take the most endorsed file on Skyrim Nexus as a case study, SkyUI, we see it has 32,501 endorsements and 1.01 million unique downloads. So of the 1.01 million individual people who have downloaded the mod, only 32,500 have endorse it. As a percentage that means only 3.2% of people who downloaded SkyUI actually bothered to endorse it. Now you could try to argue that perhaps on 3.2% of people actually still use SkyUI, or actually like it, but I think you’d be very, very wrong. Using SkyUI as an example is slightly biased as they’re at the high end of the percentage scale, most other files have much lower download to endorsement ratios, and these sorts of percentages can dishearten some mod authors.

    So what can we do to try and get more people using the endorsement system? Let me think, let me think...oh yes, our bandwidth throttling system! Here’s the plan:

    We’re going to provide increased download speeds to those users of the site who make use of the endorsement system extensively. If you have endorsed 75% or more of the mods you’ve downloaded over the past 30 days then we’ll give you a speed increase. Say, an extra 250kb/sec. But wait, there’s an obvious problem with this, isn’t there? If you say to people “endorse all your mods and you can download faster” what are they going to do? They’re going to endorse all the mods irrespective of whether they think the mod was worth endorsing or not. I can see lots of mod authors liking THAT idea, but I don’t. I want the endorsement system to at least have some sort of meaning to it, rather than as a means to an end for faster download speeds. Case-in-point, Valve added a badge to Steam that you could get only if you rated mods on the Steam Workshop. And you needed this badge in order to become a “Pillar of the Community”, whatever that is. Turns out people care about this stuff, so what happened? People (some of whom didn’t even use the Workshop) had to go to the Workshop and find a mod to rate before they could get the badge. So they went to the front page, rated any mod they could find and left. What’s the freaking point in that? The result was a load of mods that were already on the front page for being popular now being even more popular because people were rating the mods without even playing them. Facepalm moment.

    So what are we going to do about it?

    We’re going to change the endorsement system. You will now either be able to endorse a mod, or abstain from endorsing it. What’s abstaining from endorsing? Basically it’s a conscious choice you’ve made that means “I don’t think this mod deserves an endorsement, but I’ve still used the endorsement system”. It is NOT the same as a thumbs down, or a down vote, or anything of the sort. The total number of people who have abstained from endorsing a mod will not be displayed anywhere on the sites, and it’s only for our records. Why? Because if you download 100 mods over the course of 30 days and you only think 50 deserve an endorsement by your own standards, then it’s unfair that you shouldn’t get a speed boost just because you haven’t hit a 75% endorsement rate. So if you endorse 50 mods and abstain from endorsing 25-50 mods, you’ll get the speed boost. Make sense? I hope so, I’ve just written 2,700 words non-stop and even I’m struggling to understand what I’m writing.

    So now we’ve got all sorts of different download speeds you can achieve based on the choices you make on this site. And I like offering choice. So lets run-through the scenarios and the download speed limits that will be provided based on your choices:

    • You block all ads, you don’t use the endorsement system, 750kb/sec
    • You block all ads, you use the endorsement system, 1MB/sec
    • You do not block ads, you turn off the site skin ads, you don’t use the endorsement system, 1MB/sec
    • You do not block ads, you turn off the site skin ads, you use the endorsement system, 1.25MB/sec
    • You do not block ads, you turn on site skin ads, you don’t use the endorsement system, 1.25MB/sec
    • You do not block ads, you turn on site skin ads, you use the endorsement system, 1.5MB/sec


    We’ll try and add some bars/meters/notifications to the UI of the site in your member area and when you’re downloading that tells you what limit you’re currently hitting and for what reasons. It should also help to inform the myriad of people who don’t read these news articles what’s happening, and what they can do to increase their download speeds.

    I know that all this talk of incentivisation through bandwidth throttling has a few gaping holes in the plan. What if you’re on dial-up, or 1MB/sec download speeds are a dream to you? All this talk is pretty pointless huh? Yeah, it is. Sorry about that. What if you don’t care about download speeds? Well, yeah, pretty pointless too. But I wanted to look for ways that we could incentivise aspects of this site for you, and I’ll continue to look for ways to incentivise things in the future. This is the first step, I’m sure, of many.

    Talking about money, ads, adblockers, penalties and incentivisation can often leave a bitter taste in your mouth, but it’s one of these articles that I think can be quite informative. It’s often very hard to look at arguments from angles other than your own, and I hope that you can see my side of the argument, even if you completely disagree with it.
  • 29 January 2013

    Blog Piece: Nexus plans for stability and infrastructure improvements. Oh, and money.

    posted by Dark0ne Site News
    Yes, I’m sorry, here’s another one of those 5 page, 2,500 word rambling nonsense blog posts I like to make from time to time to show you that I’m not dead and we’re still moving forward. If you have no interest in servers, money or talking about my narrow views on corporate greed, then I would suggest skipping this one!

    We’re about one month into the new year now and I wanted to share with you one of our main priorities for this year, as it’s as important to us as it is to you.

    Perhaps one of the Nexus’s biggest pitfalls since its inception has been the stability of the service. I don’t have any uptime statistics on hand to report on but I think it’s pretty safe to assume that we’re not hitting the 99.97% uptime that most big sites try to aspire to. Not only is it annoying because you guys can’t access the site, or particular services on the sites (like the downloads) at random points, but it’s annoying for the staff to be awake at 3am in the morning restarting services and troubleshooting database crashes, and it’s annoying for me to be running a service that isn’t 100% reliable. Moving forward I’d like the Nexus to be taken seriously by game developers, and it’s hard to be taken seriously when you can’t guarantee service.

    It’s not as though other sites and companies out there don’t have reliability issues. I’ve been locked out of DotA 2 many times recently because the Steam servers have been down, for instance, and that’s from a multi-billion dollar company like Valve. But the problem with the Nexus is that it’s a regular occurrence.

    I can attribute this problem to all sorts of systemic issues right through the Nexus, from the way I’ve set up the business to the way we’ve run the site and prioritised things. It’s not as simple as pointing the finger at the lack of server resources, or hardware failures, or the code, or the DDoS’ers, or being a victim of our own success or any one thing, it’s a multitude of things. But the highest priority of this year is to resolve this issue and make the Nexus as stable and redundant as it possibly can be.

    So lets take a look at some of these systemic issues and then I’ll explain what we’ve been doing, and what we will be doing to make the situation better and ensure the Nexus is future-proof for the foreseeable future.

    The largest factor of all with the stability issues has been the sheer popularity of the Nexus sites coupled with having an inappropriate server architecture to accommodate the demand on sites that are extremely database (and ergo, resource) intensive. I’ve blogged many times on the popularity of these sites and the difficulty in keeping them up with the load placed upon them, so I won’t bother to go into the numbers again. So you’re thinking, “OK, why not buy more servers then?”. The answer isn’t in needing more servers, the answer is in needing to restructure the architecture of the servers and network we currently have so that the combined resources of all the servers can be used to keep the sites going.

    Right now we have a situation where we have lots of lower traffic sites (Far Cry, Neverwinter, The Witcher, Morrowind and so on), some high traffic sites (Forums, Oblivion, Fallout 3, New Vegas) and one super mega ridiculous traffic site (Skyrim). Typically speaking almost every site on the internet can fit onto a single powerful dedicated server. Depending on the size of the sites you can even fit hundreds or thousands of normal sites onto a single dedicated server. We have 6 servers dedicated to just serving the Nexus sites (not the file servers, we’ve got 12 of them!). The problem though, is that Skyrim Nexus, and the forums, are not normal sites and they’re at a point where they can no longer fit on one single dedicated server. Similarly we’ve upgraded the hell out of the servers so we can’t make them any more powerful than they are now.

    We’ve reached this point where Skyrim Nexus has outgrown being able to run on a single super-powerful dedicated server, so how do we resolve this issue? The solution is in server clustering, which is a technology that lets you pool together the resources of multiple servers to act as one super mega server, much like SLI allows you to connect up and combine multiple video cards in your PC to dramatically increase your frames per second. Unfortunately server clustering isn’t as simple as connecting an SLI bridge connector to your video cards. It’s a lot more complex.

    Server clustering is not only complex, it’s also expensive. We have 6 web servers at the moment. We can’t just flip a switch in the servers we currently have and turn on clustering. We’ve got to buy completely new servers, set them up for clustering and then transfer the network on to these clusters. That means running our current setup in parallel with the new one until everything is transferred which means paying for the original 6 servers plus the new servers we need to buy to form our clusters. That’s a lot of money.

    And therein lies another systemic issue with the way things are setup. Money. The Nexus sites have remained completely independent; free of corporate interest and investment for its entire 11 years and it shall remain so for the very foreseeable future. The only investment these sites have had was the initial £10,000 I chucked in to the sites when I rebranded the sites as the Nexus back in 2007. I’m the sole owner and sole decision maker of the sites. There’s no outside interest, board of directors or investors pulling the strings behind the scenes. Similarly no game developers have any influence or sway over me. The buck stops at me.

    If I wanted to I could make a business plan (I don’t have one, by the way) and go to Silicon Valley, pitch the idea to a load of private and angel investors, secure (potentially hundreds of) thousands of dollars in investment money and make a proper business out of it like many gaming sites and networks have done over the past few years. However, I then become answerable to shareholders and investors who are looking for a return on their investment as fast as possible. To be frank, F’ that.

    Similarly it’s just me and 4 other programmers working on the Nexus. We have absolutely no one doing ad sales. I mean it, we have no ad reps at all. Others in the industry gawp at such an oddity. That’s why the ads you see on the site (if you don’t block them) are pretty crap, and in return we get pretty crap rates. While other networks have entire ad sales teams securing them crazy $1-$10 CPM rates on their ads, we don’t. We don’t get anywhere near that. So why don’t I hire some ad reps to better sell the inventory and use that money to pump it all back in to the sites? The reasoning is very similar to my private investment reasoning; when the focus of your business is on increasing your ad sales, and on ensuring a prompt ROI to your investors, you begin to lose sight of what your original goals were and instead focus on one very simple goal: making money. And money isn’t what I’m doing this for. Indeed, if money was my aim I’d be doing all these things I just mentioned, because the Nexus would be a cash cow. Case-in-point; I know sites that have 5-10 programmers working for them and 25-50 ad sales reps. Yes, that’s a 1:5 ratio of people working on content to people working on making money. To me, it’s crazy to have more people working on selling than actually improving and producing the content that sells. But that’s business for you, and I’m not a good businessman.

    What this all breaks down to is limiting the stakeholders in the Nexus. Right now you guys, the people who use the site, the mod authors, the downloaders, the people on the forums, YOU are the Nexus site’s biggest stakeholders. If I don’t appease you then these sites cease to have a point. If I seek private investment, or start directly selling the Nexus site ads then my biggest stakeholders become the shareholders and the advertisers on these sites. My focus gets shifted from serving and pleasing you, the users, to serving and please people who have no interest in you. And the point of the site changes from being about modding to being about making money. That’s not what I want at all. There may come a time in the future when direct ad sales and private investment are exactly what the Nexus needs, but that time is not now.

    You guys are really, really good when things go wrong on these sites. By and large the reaction is often tame and supportive rather than stressed and raging. I like to think it’s because you know we’re not some corporate mega-money machine that’s cutting costs by cutting corners, but just 5 gamers trying to provide the best service we can. I don’t want to change that, because being greeted with “Ah that sucks, I hope you can fix it soon! Good luck!” is better than being attacked with “WTF this is the worst pile of crap I’ve ever used and you should be ashamed” when something goes wrong.

    I want to retain that focus on you guys being the primary stakeholders in the future of the Nexus, which means it takes a lot of monetary planning and saving to buy more servers and invest in expensive technology like server clustering while other sites can simply throw their private investment resources or ad sales money at the problem. That’s why it takes a long time. Avoiding private investment and direct ad sales is a conscious decision that isn’t without its pitfalls, but one that I think is worth it to retain the core values of what these sites were set up for in the first place; to provide mods authors with an easy platform to share their work with others that will stand the test of time.

    So setting up server clustering is currently one of our biggest priorities, and we’ll be setting that all up in the very near future, but in the run up to all this we’ve spent (and are still spending) some considerable time right now focusing on the software side of things.

    Over the Christmas period, while the “normal” members of the staff were enjoying a forced two week break, Axel was working on an error logging system for the Nexus.

    One of the most annoying aspects of bug hunting and troubleshooting is when someone leaves a comment on one of these news articles, or on the forums, or on the tracker that something is broken. Typically it will go something along the lines of “Downloads are broken at the moment”. To which my response is “.......” , followed by much hair pulling. Downloads are broken? What downloads are broken? On what site? What files? Is it all files or just one file? Is it only happening on one Nexus site or all Nexus sites? Is it just small files or large files? What error are you getting? Is it happening 100% of the time or just some of the time? What browser are you using? Have you tried using another browser? Have you tried turning your PC on and off again? Have you tried logging out and in again? Did downloads ever work for you? Have you installed any new browser plugins, firewalls or anti-virus programs recently? What time did this happen? These are but some of the questions we need answered to actually troubleshoot the issue, especially if all the staff try downloading and it works fine.

    What I wanted was a system that would aggregate and parse all the error logs the servers produce and present it to the staff in a system that can help us easily pinpoint not only errors and problem areas of the site, but also pinpoint specific times when the sites are worse than others to help us troubleshoot the problem. Typically the error logs that servers produce are all flat-file text documents. Line after line after line of errors with timestamps that can run up to gigabytes in size. It’s extremely hard to make use of these error logs without having a system to properly display the information, and there’s nothing worse than being told something isn’t working when it works for you and wondering if it’s affecting just one person, 1% of people 25% of people or even more. With the error logging system we can now see that “wow, yes, at 10am today we had 5 times more errors than we usually do”. It’s helping us to investigate things more and we’ve already applied numerous hot fixes to the sites over the past month that have patched up errors and slow areas of the site.

    Similarly at this very moment we’re working on some more improvements to the downloading system for both manual downloads and downloads through NMM. Right now, if one of the file servers has hiccupped it can be a real pain in the ass trying to download something. These hiccups generally only last for minutes at a time, but during that minute it can make it hard to download any files, especially small files. With that in mind, we’re going to present the file server selection screen on all files now, irrespective of size. If a file server is down, you can quickly select another one to use. We’re also trying to implement a seamless redirect system incase you choose a file server that isn’t working for whatever reason. If the file server you choose isn’t working, the site will simply try another one until it finds a file server that is working. You won’t really notice a difference (except far fewer, or no errors at all!), although if you typically get fast speeds on only one or two servers you might get slower speeds as your download might be served from a different file server from the ones you normally pick, if they’re down.

    This concept of seamlessly being moved to a server that works is very similar to our plans with the sites and servers in general. Right now, if there’s a hardware or network failure on one of the servers a Nexus site is on, that Nexus site becomes unavailable. Once our full clustering solution is done we’ll have a load balanced, redundant solution that means all sites are being served from all servers. If one server goes down, the other servers pick up the strain but the sites still work. It reduces the bottlenecks and also reduces our single point of failure problems. And finally, clustering restores the status quo by making “buy more servers” the viable solution to our strained server issues. If the network needs more power you just tack another server in to the cluster and you’ve boosted the resources available to the network. That’s not possible with the current system. So for me, this is quite exciting. For you, it’s more like “I don’t care, just make it work”.

    I think I’ve gone on for long enough now. When I wrote my blog piece on Nexus development and expansion philsophy I was interested when I got a lot of emails and messages from people who were surprised I thought that hard about the process. I like to use these blog pieces to indeed show you that yes, I don’t just sit around all day watching stocks go up and down on my monitors and playing DotA 2, but I think pretty damn hard about this network. The choices I make aren’t just knee-jerk “oh, I guess we’ll just do that then” solutions but plans that have been made out and expanded upon over a long period of time in consultation with others. We’ve wanted to do server clustering for years now (indeed, I mentioned it in that first YouTube video I did), but we’ve only now been in a position to actually afford it. And that’s why I’m excited, even though the topic is pretty boring.