While we have our cluster setup looked in to we’ve been busy looking in to some other areas of the site we’d like to improve before we “centralise” the Nexus in to supporting mods for all games.
We’ve come up with a new file adding wizard that’s usable on all sites. Indeed, when we finally get around to centralising our database you’ll be able to add a mod for any game ever made to the site through this one file wizard, but for now it’s setup to work with the sites and games we already support. The idea is that the new file wizard will take you, step-by-step, through the process of making a complete file page for your files. While in the past you would simply add a few details and then be expected to add things in later, now you’re taken through every step of the process, with 9 steps in total.
You can skip steps that you don’t want or don’t need to do (like required files), or skip the wizard entirely if you so wish, but I’m hoping that this new process will at least introduce new mod authors to all the file page features we currently support so they can provide as much relevant information to their users right from the start while also providing veteran mod authors of the site with a quicker way of get their pages set up on the site without having to keep going backwards and forwards.
We’ve also added a new set of options to the wizard that allow you to better categorise your files. It actually makes use of our tags system, but in a much more obvious manner, in an attempt to better support our content blocking system, which I’ll now discuss.
Recently we had a couple of nude mods show up in our hot files on Skyrim Nexus. It’s not a particularly rare occurrence by any stretch of the imagination, but what was a bit more rare was the content of one of the mods. A penis. EEK, A PENIS! Lets get this out the way. Please repeat after me: penis penis penis, vagina vagina vagina. Got that out your system? Good. While we get the occasional rant about big boob mods or general female nudity on the sites, nothing sparks an absolute outpouring of pent up closet-case emotion like the showcasing of a penis mod on the site. Indeed, no less than 6 people sent me a message to inform me of their outrage of bearing witness to a penis mod on the sites. When pressed for clarification on why vagina mods were ok but penis mods weren’t the conversation went ominously silent. While I’m no expert in the field let me provide some advice: if seeing a penis mod throws you into a fit of quizzical fury you might need to explore your own sexuality a bit further. Don’t repress it, explore it. You’ll save yourself lots of therapy payments in the future.
With that said, I can completely understand why you might not be interested in seeing a penis, or a vagina, or boobs, or scantily clad men or women, or translations, or lore-breaking files, or save-games or any number of various types of content (indeed, the issue isn’t in not wanting to see the content but the way in which people get so comically worked up over the issue). While we can’t provide a 100% success rate method of removing this content from your viewing we can try our hardest to tailor the mods you see to your specific preferences.
We’ve had content blocking available for a long time. It works by blocking files that have been tagged with specific tags. If you block the “Sexy/skimpy” tag you won’t see any mods that are using that tag. There are a few pages that are the exception to that rule, the hot files originally didn’t block this content but it now does (we fixed it recently) and the new today/new recently pages don’t. However, the category results/search result pages will block this content. We even go one step further and tell you that you’re currently blocking content and you’re only seeing X number of files as Y number of files have been blocked from your viewing, with an easy link to see the results with your previously blocked content showing as well. The systems are in place, you just need to use them.
There are two inherent problems; (1) not enough mod authors tag their files and (2) people complain about files not being tagged but then don’t tag files themselves. Your profiles have a statistic on them called “Files tagged”. It tells people how many files you’ve successfully tagged. If you complain about files not being tagged properly and your “Files tagged” statistic is 0, you’re a part of the problem and not a part of the solution. Your complaining about the lack of tagging doesn’t fix the issue, your actual tagging of files does. So next time you see a nude mod that hasn’t been tagged properly, “doing the right thing” isn’t complaining about it, doing the right thing is actually tagging the file yourself so that others don’t have the same problem you did.
With that said we’re trying to do our part to make these features more prominent. In the new file adding wizard we’ve hand-picked some tagging selections that are known to be the most regularly blocked tags on the sites. You’re asked “Tick any of the boxes that apply to your file. My file...” and on Skyrim Nexus, for example, options include “contains nudity”, “contains sexy/skimpy outfits”, “is a translation”, “is for male characters”, “is for female characters” and so on. (Note: as of writing I haven’t actually gone around and added these, but they’ll be up within an hour of me posting this article). Ticking these boxes will automatically tag your file with the relevant tag.
On the user end, we’ve added these same tags to the file search page under “Attributes”. Each tag is given a tick box. If the tag is ticked then you’ll see files using that tag. If you untick the box you will no longer see files using that tag. It works exactly the same as our content blocking system but it’s just much more obvious, and provides a link to the full range of tags you can block on the sites.
The aim is to make content blocking both more relevant, by asking direct questions about the type of content your mod contains, and also more prominent, by making it a part of our extensive filtering system that should help you to drill-down in to the file database and discard the types of mods you would never be interested in anyway. While this system was always present on the site it was never this prominent.
You’ll also notice the new file wizard has some new options for categories. This is more of a precursor for our centralisation work as I think most of the sites are pretty set in their categories now, but I’ll quickly run through it. You now have three category options. You must pick from a pre-set list (as before) but you can also suggest your own category that you think would best suit your file, or you can select from a pre-set list of what others have suggested. What’s the point of this? With current sites there’s not too much of a point to it as there’s already plenty of usable and relevant categories, however if we start supporting new games we want to be as automated and flexible as possible. The idea is if you’ve got a texture replacer mod for a new game, you add the game yourself and the site will automatically setup the game site along with a “Miscellaneous” category (all games have that category in them, current and future). You select the “Miscellaneous” category for your file but you suggest a “Texture replacer” category. While the file will begin in the Miscellaneous category if an admin agrees with the suggested category any files uploaded to the “Miscellaneous” category but using that suggested category will then be moved over to this new suggested category. The suggested category becomes a new category in and of itself. It sounds complicated but it’s really not.
Alrighty, I’ve gone on long enough. I hope these new features can be put to good use and if you think something isn’t working properly then let me know.
Just two mods for you in this video, but they are definately worth checking out.
Thumbnail image for this video is 'Spriggan Gone Evil' courtesy of andreamkall.
Download links for reviewed mods:
Armor of Intrigue
Mod preview videos:
Skylight 11 (Skyrim Mod Preview) : Spell Crafting for Skyrim
Skylight 12 (Skyrim Mod Preview) : SkyUI v4
I’m about ready to throw a load of paper in the air, scream from my balcony window, set fire to some underwear and go on holiday to sunny Mexico for a couple of weeks and pretend I don’t run these sites. Dark0ne? Nope, no idea who that is (and what a stupid name). Don’t worry, I won’t though. The girl can’t get time off work and I promised I’d house someone next weekend. Alright, alright, I wouldn’t do that to you.
So what’s going on at the moment? SHITE, that’s what! Where to start? Since our failed attempt at setting up the MySQL Cluster last Thursday and Friday we reverted back to our old setup. Naturally the weekend was bad, as they have been for the past month or two, then during last week things went back to being fine, and then, again, this weekend, it was awful. 502 errors, 504 errors, I’m sure you know what I’m talking about. I’ve had enough of that, I’m sure you have too. It may be frustrating for you, it’s freaking debilitating for me.
There’s various issues right now that have reared their ugly head and require immediate remedying. Remember when Skyrim first came out and we were struggling under all the load? The sheer amount of people wanting mods was crushing us. Back then we were getting around 6 million page views a day. If you do the math that’s around 69 page views a second. Typically traffic is meant to go down over 20 months, but nope, we’re still pushing around 5.2 million page views a day, or 60 page views a second. Of course, we’ve made loads of optimisations since those days, but there’s also a big difference between back then and right now; back then our database was tiny. It didn’t have 30,000 files, or 300 million downloads in the database. Searching a database of 300 million rows for your user ID is, naturally, going to take a lot longer than searching a database of 1 million rows for the same user ID. So while traffic has gone down, slightly, our database has grown by many, many, MANY orders of magnitude which has slowed things down. It’s made our jobs difficult as we constantly work between new features, tweaking the servers, the new cluster setup and what not. Frankly, it’s a pissing nightmare and our jobs right now are most definitely not fun. They haven’t been for quite some time.
This is what the Cluster setup was meant to fix for us not only by providing us with a hell of a lot more power for us to make use of but to also make the concept of increasing that power whenever needed that much easier. Instead of trying to eek out every last drop of performance from the servers with config/ini tweaks we could simply buy a new server, tack it on to our cluster and hey presto, we just increased the power available to us by X amount, quickly, easily, efficiently, freeing up time for us to work on making the sites better, rather than simply making the sites work.
I’ve been on the lookout for someone who knows what they’re doing to help us out with our cluster setup. Unfortunately the MySQL Cluster experts, certified in the job, cost an astronomical amount of money to hire on. I got quoted $200/hour from 3 separate companies. If you consider it takes around 12 hours to import our forum database in to the cluster, that’s $2,400 just to do that one simple task. I imagine it’ll need at least 50-60 hours spent on it, and $10,000 is quite an expense. Even then there’s no guarantee it will be completed within that timeframe and I can’t write a blank cheque. The cluster is our salvation and our future, we just need it to work!
Today Skyrim Nexus was absolutely awful, 502 and 504 errors galore. The reason? One of our software licenses decided to think it had expired, when it hadn’t (due to expire in 2014) and brought down the server with it. cpnginx, take a bow, you’re freaking badly made! We’ve got that fixed again and now we’re diagnosing yet another issue.
Our database isn’t the only thing that has grown in size over the past 2 years; the number of people using the Nexus Mod Manager has as well. We’ve just passed the 2 million unique users mark for NMM, and, similar to issues we had 6 months or so back, we’re having to tweak things again to be able to cope with the demand. We’ve actually turned off the NMM web services this evening to run some tests. Getting a “file does not exist” or “server unreachable” error when trying to download through NMM? Sorry, that’s us. Also, sorry we can’t give you a better error message than “file does not exist” when, actually, it does exist, the services are just down. That’s just stupid. I’ll get that put in to the next version of NMM so that when we take down the services for whatever reason you’re informed properly instead of thinking something is actually broken and come to the forums looking for blood. We’ve found that turning off the NMM services makes Skyrim Nexus more than good to browse; I was able to click through 20 search result pages in a minute, a freaking enigma in this Nexus day and age. If we turn the services back on everything slows down to a crawl again. Safe to say NMM is causing problems for the sites, so we’ll be looking into that as soon as possible as well. I’m going to leave the NMM services off until we go to sleep tonight in a couple of hours time. Browse Skyrim Nexus, isn’t it better? Isn’t it MUCH BETTER? That’s what we want it to be like all the time.
And finally I received some “good news” last week, depending on how you want to view it. When our file servers were hacked a few months ago and some downloads were serving malware instead of mods we put in extensive work to not only “harden” the servers but add a ridiculous amount of monitoring to the servers. If you so much as look at our servers in a funny way we’ll be notified about everything we can possibly know about you. NSA, eat your heart out. A couple of weeks ago we were made aware of some breaches to two of our servers again. We were notified automatically and immediately about the breach and took the servers down within 5 minutes of finding out. We spent a ridiculous amount of time trying to work out how the hacker(s) had gained access to our servers. When we exhausted every option we got in contact with the provider for the servers. Turns out we wasted about 100 hours of our time as the breach was in their systems, and not ours. I cannot tell you much about the topic as it’s under investigation with the FBI, but the attack is thought to originate from the Ukraine and it was specifically targeted at the Nexus and one other site/network at the provider. This, frankly, was a relief for us. Not because we were hacked, of course, but because it wasn’t “our fault”. There was nothing we could have done, or added to the servers, to prevent this attack other than not use this ISP altogether. This ISP has been good to us over many years, we had no reason to doubt their competence and they are a major international player on the world stage. The fact the hacks came through this ISP have meant we can sit back and let them stretch the full force of their contacts at the FBI and local crime authorities, rather than us trying to flex our petty influence. This tells me a couple of things: (1) these people targeted us specifically but (2) they couldn’t hack our servers, so they instead hacked our ISP instead to get to us (3) we know when something bad has happened and can act very quickly to prevent anything reaching you (4) our ISP acts fast to rectify the problem on their end. It’s not exactly a happy story, but it was a relief for me at least.
Before I sign off let me just say this. We know there are problems with the performance on the site. It’s very hard to miss. I appreciate all the people letting me know about the problems through various mediums; the forums, support tickets, email, etc.. A few people get pissed off that I don’t respond every day to the posts on the forums on the topic. The inherent problem is that when I respond, or even make a news post like this one, it gets buried within 5 posts and you get Joe Blogs coming along who couldn’t be bothered to check the news or read all the posts in the thread asking the exact same question for the exact same problem, normally across 5 or 6 different threads. This is annoying, and if I spent my time responding to every single post like this I’d have no time for anything else. So understand I most definitely know about these issues and silence from me doesn’t mean I’m disinterested, silence means we know about the problem, we’re working to fix it, and I’ll let you know as soon as I know more about the problem myself.
PS. You CAN still use NMM even without being able to login. Click the "Offline" button and you have access to all your mods. You can even install mods, you just need to use the manual approach (and NMM supports dragging and dropping).
ENB mods have been around since the start of Skyrim and yet several people have not tried them because they either don't know what they do, or are nervous about installing them. So I have started a new video series aimed at exploring the world of Skyrim ENB mods. In this, the first video of the series, I look at a popular classic, 'Sharpshooters Extreme Graphics ENB', and discuss what you get, and the price you pay to get it.
For an overview of the aims of this series see this video.
Watch the installation video for this mod here.
Download link for mod:
Sharpshooters Extreme Graphics ENB
The SkyUI mod may be the most endorsed mod in Nexus history, but they keep on making it better. Here is a quick preview of a new feature that is coming soon in version four.
Thumbnail is 'Adventuring' courtesy of andreamkill.
Download link for mod:
After 2 days and no sleep (Axel has been up since the early hours of Thursday morning), we’re back to fully operational. With a heavy heart I also have to report that we couldn’t get our clustering setup to work properly. While the past 2 days of limited service haven’t been for nothing as we’ve learnt that we can’t do it without professional help it does mean that we’re still running on the old setup which is disheartening to say the least.
Things should be back up and operational again (as though nothing has changed), and while I’m sorry for the downtime and inconvenience caused I’m sure you can understand our wish to try and attempt doing this on our own rather than forking out the thousands of pounds that will now be necessary to get this looked into by a professional database consultant on our behalf.
We tried and we failed so now we have to move on and hope that we can rely on some certified engineers to do the work for us.
Thanks, as always, for your patience during this time.
It seems like an age ago that I was here writing about our plans for stability through server clustering. It was a long time ago, a much longer time than I was anticipating or hoping, but we’re as ready as we’ll ever be to roll it out.
As a part of the original announcement I did say that we’d probably have some substantial downtime as we moved the site databases over to our new architecture. That downtime (perhaps one of a few planned downtimes) will be tomorrow. All day tomorrow in fact.
All is not lost, however. It’s not going to be a total downtime of all the sites, but rather this downtime will simply be for the forums. You might now be saying “ooooh that’s fine, I don’t use the forums”. You probably do use the forums, you just don’t know it, as the forums are used for file commenting, new user registrations and uploading new mods to the site. So during this downtime not only will you not be able to use the forums themselves, but you won’t be able to post new comments/topics, add new mods to any Nexus site or create a new account. You will still be able to browse the Nexus sites for mods and download as normal.
We’re expecting this downtime to start around about 10am GMT tomorrow and last all day, and then we’ll likely be spending most of Friday fixing problems that might arise and tweaking everything for performance. So it could be a rocky few days coming up, but the long term benefits should outweigh the short-term inconveniences.
Once the forums are working on the cluster we’ll be moving the larger Nexus sites over. Our smaller sites are already using the cluster (anything with less files than Dragon Age/Fallout 3). These sites might require some downtime but we’ll inform you before any work is scheduled to start.
Skyrim Mod Sanctuary 68 : Throwing Weapons, Gladiator Armor - Spartacus, and New Animation - Power Thrust
Ever wanted to throw weapons, fight like an infamous gladiator or just stab people viciously with a pointy thing? This video might well be for you.
Thumbnail image for this video is "Dragon in the Mist" courtesy of symplexity.
Download links for reviewed mods:
Throwing Weapons - Death From Afar
Throwing Weapons Redux - Beta
Gladiator Armor - Spartacus
New Animation - Power Thrust
Mod preview videos:
Skyrim Heart Rate
One of the most customizable armor mods ever made, a great selection of weapons, and a mod that will help immerse yourself in the game.
Thumbnail image for this video is courtesy of ZeroKing.
Download links for reviewed mods:
Warrior Within Swords
Mod preview videos:
Imaginator - MCMenu
Sound Propagation and Footsteps Sounds Overhaul
We came across a bug today that meant when users downloaded files through NMM the download was not being counted as a “Unique download” but it did add to the “Total downloads” counter. For those who don’t know, unique downloads are counted once per user, per file, per page. So if you download the same file 10 times it adds 10 to the total downloads counter but only 1 to the unique downloads counter. It’s a more accurate way of seeing the true number of unique individuals who have downloaded your files.
We’ve been able to run a script to recount all the unique download counters and I think some of you will be pleasantly surprised how high your unique download counters have jumped (I’ve seen some that have doubled, or even tripled, in count).
This bug only affected files with NMM enabled and those who downloaded the files through NMM.
Jump to page