Notifications
Notification preferences
notifications_off

No unread notifications right now

You're up to date
See All

You're up to date

File information

Last updated

Original upload

Created by

Patrorex

Uploaded by

Patrorex

Virus scan

Some manually verified files

Tags for this mod

Tag this mod
Current section

About this mod

Automatically finds and deduplicates identical mod files across folders using safe hardlinks. Saves disk space by replacing duplicate assets like .dds, .nif, and .bsa without breaking mod setups. May have many false matches - always re-run Wabbajack after using. Will improve with future updates.

Permissions and credits
Changelogs
📌 Note on Antivirus & First Launch
When you run my exe files for the first time, Windows Defender may show a “Windows protected your PC” warning (blue dialog box).
This happens because the EXE is unsigned and from an unknown publisher - a common case for new or small utility tools.
🟦 To proceed:
Click “More info” → then click “Run anyway”.
✅ All my tools are safe. Nexus Mods scans all files, including .exes, before approval and verifies them using VirusTotal.
You're always welcome to manually scan the file for peace of mind.

📌 Developer Note
I create EXE-based optimization tools, and new releases may trigger temporary flags from antivirus software. This is expected for unsigned applications.
👉 Check out my other tools on Nexus Mods

I'm focused on building Skyrim optimization tools, and using them myself I was able to save 207 GB from old archives and 130 GB from hardlinking duplicate files (Just from 2 heavy modlist) in a Wabbajack modlist - that’s a total of ~330 GB saved!
In the meantime, you can check out my other tools here and help your poor SSD breathe a little easier. 😄

This tool is still in early development. It now performs hashing few files to avoid false positives while other files use filename with size for which there is still a small (~1 %) chance for false positives. However, the tool remains safe to use with Wabbajack modlists, since Wabbajack will automatically verify and repair any incorrect or missing files. For now, the tool is recommended primarily for Wabbajack-based setups (as we can fix false match with Wabbajack)

Please share your feedback and test results on Nexus to help improve future versions!

Note: Only works for folders on same path.
Due to limitation of NTFS hardlink


🥐 Are you a modlist junky just like me? Then here’s a sweet roll for you!

Tired of juggling multiple modlists and crying over not having space for just one more? This tool was made exactly for that struggle.

So, Modlist Installation Deduplicator is a lightweight utility for modlist builders and users that identifies and replaces identical files across mod folders using hardlinks saving gigabytes of disk space without affecting mod structure or load order.
This project is actively evolving, and community feedback is highly encouraged to help improve accuracy, performance, and compatibility.

🧠 Why Use This?

Large modlists often contain exact carbon copy of Skyrim files and duplicate files (textures, meshes, BSAs, etc.), especially among modlists that share common mod dependencies.
This tool can:
  • Saved 19 GB with two 57 GB sized modlist shown in test results below.
  • Save larger size on forked modlists (e.g., NGVO as base, with forks like Lorerim reusing maybe 150+ GB of the same assets)
  • Even with other heavy modlist, you could see lots of shared files.
  • Work safely with Wabbajack, which will auto-repair any mismatches
  • Handle .dds and .nif files more accurately using hashing
  • Fully reversible - if anything breaks, Wabbajack will re-download just the affected files

🔧 What It Does

  • Scans one or more modlist folders
  • Identifies duplicate files using filename + size
  • For .dds and .nif files, uses hashing instead to avoid false matches
  • Replaces matching files with hardlinks to a single copy
  • Works seamlessly across Wabbajack modlists (Recommened)
  • For MO2 setups, or any standard mod folder structure (Not recommened few files may or maynot be falsly linked). If you know how to check logs, you can identify and fix any issues manually. Otherwise, it’s best to wait for future updates as the tool becomes more polished for non-Wabbajack modlists.
  • Designed to preserve both folder layout and load order hierarchy

📁 File Matching Process

We use a hybrid deduplication method with three main steps + one step for users to validate the process:
  • Scanning: Recursively scans files with supported extensions: dds (hash), .nif (hash), .bsa,  .osd, .tri, .pdb, .wav, .bik, .mohidden, .cgid
    Note: Can customize in Configure Settings in main window.
  • Each file is evaluated by size and, if applicable, hash.
  • Matching: Most files are matched using filename + size.
  • For .dds and .nif, we use hash comparison only - allowing us to link identical files even with different names.
  • Hardlinking & Verification: Once duplicates are found, they are hardlinked to save space.
  • Then you should run Wabbajack. It will verify everything when launched, fixing anything mismatched.

🛠️ How to Use the Tool

Step 1: Open Configure Settings from the main window.


Step 2: Decide how you want to use the tool:
Option 1: Just Getting Started - If you're unsure what to do and don’t modify your output folders:
➡️Use the Default profile in Configure Settings.
Option 2: Maximize Space Savings - If you don’t modify output folders and want to save as much space as possible:
➡️Use the Aggressive profile. This includes extra extensions like .bto, .btr, etc., for better space savings.
Option 3: Custom Output / Special Cases - If you modify output folders / mods that you added and don't want to hardlink
➡️Choose any profile and modify the setting as you see fix also add to exclude those folders.
💡 Note:
If manually adding exclusion paths is too much hassle:
1) Temporarily move your output folders (any folder that you don't want hardlinked) elsewhere before running deduplication.
2) After the process is complete, move them back. This ensures they won't be scanned or hardlinked.

💡 FYI:
These are some of the common file types used for hardlinks in modlists:
.dds(hash), .nif(hash), .bsa, .osd, .tri, .pdb, .wav, .bik, and few more
Files marked with (hash) are identified using hash-based comparison to detect duplicates. The rest are matched using filename + size. If you know of any other extensions that should be added or removed from the list, feel free to share - your input helps improve the tool for everyone!

Post-Run Mode (For Existing Modlists)

✅ Step 1: Manage Game Root (Post)
This will deduplicate your actual Skyrim game folders.
  • From the main window, select “Manage Game Root (Post)”
  • A new Game Root Manager window will open
  • Add the Skyrim folders you want to deduplicate (one clean folder will be used as the reference; others will be replaced with hardlinks)
    👉 Example Paths:
    D:\Modlists\Ilinalta\Stock Game  
    D:\Modlists\TON\Stock Game
  • Paste or type the folder paths, then click “Add Folder Paths”

Optional (But opted by default) : Dry Run (Preview Mode)
  • Dry Run is enabled by default.
  • Click “Run Deduplication” - it won’t make changes yet.
  • This simulates the process and shows: summary eg: no. of files would be linked, space saved ,etc.,.
  • Then you can click "Run Now" button to start the actual process
  • Else if you don't want preview,
- Uncheck Dry Run, then click “Run Deduplication” to apply changes.
- The tool will hardlink predefined file types (see list in the window)
When finished, click Back or Close to return to the main window.

✅ Step 2: Manage Modlist (Post)
This will deduplicate your mod folders (used by Wabbajack Modlist's mod folder).
  • From the main window, click “Manage Modlist (Post)”
  • Just like before, paste or type your modlist folder paths
    👉 Example Paths: 
    D:\Modlists\Ilinalta\mods
    D:\Modlists\TON\mods
  • Click "Add Folders"
  • You can:
- Do a Dry Run to preview changes
- Or uncheck Dry Run and directly click “Run Mods Root Dedupe” to start without preview.
When finished, you can close the tool.

🕒 Performance Estimate
  • For a modlist with 68,000+ files and ~115 GB, the deduplication process completed in ~3 minutes on a mid-range NVMe SSD.
  • On larger modlists - say 700,000 files and 800 GB+ - you can expect the process to take 20-30 minutes, depending on your drive speed and system load.

✅ Step 3: Verify with Wabbajack
  • Once deduplication is complete, launch Wabbajack. It will verify everything.
  • If something’s off - like a mismatched or incorrect file then Wabbajack will automatically redownload and fix it.
  • Here, Big Billy Wabbajack is your guardian - ready to nuke fake files and restore order with his loyal minions.

🧪 How It's Tested

With v1.2.0:
I tested the tool on two large modlists (combined size ~700 GB with around 530,000 files).


The tool saved ~130 GB total:
  • ✅ 109 GB saved from the mods folder
  • ✅ ~20 GB saved from the game root directory 
⏱️ It took about 23 minutes to run

As shown above, the tool did its job well. Check verification info from Wabbajack:
  • For the first list, 4 mods had false matches, resulting in 6MB being overwritten.
  • For the second list, 24 mods had false matches, so Wabbajack is extracting about 300MB of files.
📝 I forgot to take a screenshot of the drive’s free space after the process, and I’ve already started moving my archives to another drive - so you’ll have to take my word for it 😅
📦 But yeah, the tool just saved me ~130 GB - and that’s no exaggeration!

Before v1.2.0 :
Setup:
Initial testing was done using two small Wabbajack modlists:
  • Ilinalta - 57.6 GB / 23,148 files
  • The Old North - 57.4 GB / 45,489 files
→ Total: ~115 GB across 68,637 files
→ Scan Time: ~3 minutes on a mid-range NVME SSD setup
Results :
• Total size: ~115 GB across 68.9k files
• Completed in ~3 minutes
• Saved 4.6 GB in mods folder
• Saved 12 GB in game root
• ~19 GB total saved for these small lists (115 GB to 96 GB)
• Wabbajack didn't find any broken/mismatched files this time






Few lists don't have many overlapping mods.
But, imagine this impact on big modlists like NGVO, which has been forked into dozens like Lorerim, Ghoulified, etc.
Even unrelated heavy modlists still share a surprising number of identical files.
That’s where this tool really shines.

⚠️ Warnings & Recommendations

  • In this version, only .dds and .nif files are hashed, so they are safe from false matches.
  • For all other file types, the tool uses file name + size as the identifier, which is fast but may result in ~1% chance of false matches.
  • This is not a problem - just re-run Wabbajack after using the tool, and it will fix any mismatches by redownloading affected files.
  • Based on current testing, only .dds and .nif files were observed to have same name + size but different content. Other file types appear safe so far.
📣 Help Improve the Tool
Your feedback is crucial. After using the tool:

  • If Wabbajack reinstalls any mods after using this tool, check the modroot_dedupe_log.txt file for clues. (Little difficult for normal users)
    Or If you're unsure, feel free to share your modroot_dedupe_log.txt file along with a screenshot of the Wabbajack UI logs - I’ll be happy to take a look and help improve the tool for everyone!
  • Alternatively - "easy but takes time to process :(", run a dry run (no linking) using the tool - it will show lines like:
    ✔️ Would link:
    These lines represent potential false matches.
  • If you can share these lines - it would greatly help improve the matching logic across various file types and modlists.

This kind of community insight could eventually allow us to skip Wabbajack re-verification entirely, making the tool fully usable for personal or non-Wabbajack modlists too.

💬 Final Notes

This tool helps more than just your SSD - it makes modding smarter. Forked or unrelated modlists often share massive amounts of files. By deduplicating them, you're saving space and helping us improve matching logic.
If Wabbajack reinstalls any mods after using the tool, check the log and share false match info (like unexpected file types). That feedback directly improves future versions - and might even make verification unnecessary one day.
Thanks for helping the whole community benefit!

🆕 Version 1.2.0
  • Fixed issue where app would crash on dedup process
  • Added Config settings to let user decide what files to hardlink , what folders to skip and what files can use hash
  • This suggestion comes from community feedback, and I’m still collecting insights. If you have knowledge about mod file structures or best practices for hardlinking, please drop a comment - it helps make the tool better for everyone.

🆕 Version 1.1.3
  • Remove .bto from hardlink list

🛠️ Planned Updates

Here’s what’s coming next in future versions of the tool:
Post-Processing Installation Mode (Experimental)A new mode to gather and hardlink all mod files from multiple lists into a clean, empty modlist folder. You then install your chosen list into this folder using Wabbajack. What Wabbajack will do is:
• Keeps only required files
• Automatically removes unused ones
• Reduces space usage and SSD/HDD wear over time

Optional Symbolic Link Support for post method:
For users who want to deduplicate across drives, symbolic links will be added (by request). Note: this is less safe - if the original file is deleted or you want to Wabbajack updates then the symlink breaks. Not recommended but hey this is also an option.

Community testing, feedback, and improvement ideas are highly appreciated.
If you try it on a large modlist, feel free to share your results!
Let’s make modlists leaner and cleaner - together.