SKYRIM
Merge Plugins xEdit Script by matortheeternal
Skyrim » Utilities
Added: 05/07/2013 - 08:24PM
Updated: 21/10/2014 - 03:40AM

3,152 Endorsements

1.75 Latest version

27,382 Unique D/Ls

37,052 Total D/Ls

203,293 Total Views

Uploaded by matortheeternal

Description

Last updated at 22:40, 20 Oct 2014 Uploaded at 15:24, 5 Jul 2013


Have a question? There's a 99% likelihood it's already been asked. The most sureproof way to find the answer that I've already typed out at least a dozen times is to use the search function on the Nexus Mods thread. I'd also recommend you check the FAQ section.




THIS SCRIPT ONLY WORKS IN TES5EDIT/FNVEDIT/xEDIT 3.0.32 OR NEWER!
I highly recommend you get the xEdit 3.0.33 beta to make use of all the awesome changes that version offers.


Will <insert mod here> work?
When in doubt, try it out.
If it works - without quirks,
Let us know, our knowledge grow!

If it fails, drink some ales,
Post a report, errors to sort!



//==============================================================\\


Old videos: 07/05/2013, 11/05/2013.

\\==============================================================//

For updates between official releases, please see the GitHub repository.




Version 1.7:
//=====================================================================================================
Finally! Version 1.7 is publicly released!

What's new in this version:
- I've introduced a dictionary file which has definitions on how well certain plugins merge.
- Files will be colored in the UI based on how well users report them to merge.
- Files will have additional information on the reports submitted by users in their mouseover
hint box.
- Mods with MCMs can now be merged thanks to a new asset copying feature.
- The second pass of copying now uses the merge method specified by the user.
- Introduction of a new script, the Merged Plugin Manager, which can be used to remove
plugins from a merged plugin or generate reports on plugins.
- Introduction of a client-server interface by which you can submit reports and get
updates to your dictionary file through Merge Plugins Report Client.jar
- There's now a stylish new progress bar while merging.
- The user will be notified when their version of Edit is out of date.
- You can now merge files and renumber formIDs when more than 128 files are loaded in
xEdit, assuming you're using xEdit v3.0.33 or newer.
- You can now merge into an existing plugin while using two-pass copying..
- Code refactored, cleaned up, various issues fixed.



Description:
//=====================================================================================================
It's been known for awhile that you can merge mod files using TES5Edit by renumbering the
FormIDs and Deep Copying them. This script perfects that process and automates additional
steps to make sure the resulting Merged Plugin functions perfectly.

To use this script, just select the mods you want to merge, run the script, type in a name,
and presto! Your mods are now in a single clean package!



The Limitations:
//=====================================================================================================
You cannot merge mods with corrupted data. These are mods that were edited using a tool like
TESVSnip, a SkyProc patcher, or other third party tool which have data which is corrupted and
cannot be copied by TES5Edit. When using Check For Errors this appears in the form of
"Invalid or out of order subrecord" errors. You can attempt to fix this by opening the ESPs in
the CK as the active file and resaving them, but this may not always work. If you merge a file
with corrupted data the resulting merged file will cause CTDs when Skyrim starts.



Installation:
//=====================================================================================================
Before installing and using this script you need xEdit 3.0.32 or newer (3.0.32, 3.0.33).
xEdit is an alternative editing program to the Creation Kit. While xEdit may not have some
of the cool features the Creation Kit has, it runs faster and allows for the execution of pascal
scripts. I highly recommend you get xEdit 3.0.33. It's faster and has the most up-to-date
definitions.

You can get the current public release of TES5Edit here.
You can get the 3.0.33 beta of xEdit here.

Once you have downloaded and installed xEdit you simply need to copy the contents of the
included "Edit Scripts" folder into xEdit's "Edit Scripts" folder. This folder is located in the
directory where TES5Edit.exe is.



Tutorial:
//=====================================================================================================
Merging Plugins:
1. Set up the plugins you plan on merging so they are contiguous in your load order.  That is, make
sure they're all in adjacent load order slots and don't have master dependencies in between each other.
Test to make sure these plugins create the correct behavior ingame before merging.
2. Extract the BSAs associated with the mods you want to merge.
3. Open TES5Edit and load the mods you want to merge.
4. Use check for errors on files to be merged and fix any dirty edits TES5Edit finds.
5. Hold control and click on each of the mods you want to merge so they're highlighted.
6. Right on one of the mods and click "Apply Script".
7. Choose "Merge Plugins v1.7" from the dropdown menu and click OK.
8. Verify the mods you want to merge are checked, and that no other mods are checked, then click OK.
9. Choose -- CREATE NEW FILE -- and click OK.
10. Enter the name you wish to use for your new file, not including the file extension.
11. After the script is finished, verify the merge worked correctly. Also verify the masters of
the merged ESP don't include the files you merged.
12. Disable the ESPs you merged from your load order and enable the merged ESP you made.


Extracting BSAs with Mod Organizer:
Mod Organizer has an option to extract BSAs for you. Right click on a BSA archive in the "Archives"
tab, click "Extract BSA", then specify the Mod Organizer\Mods\* folder corresponding to the mod the
BSA is for. 


Extracting BSAs Manually:
You want to use a program like BSAOpt or FO3's Archive utility to open the BSAs in question, then
extract their contents directly into your data folde, so the folder structure matches that which
already exists in the data folder.


Verifying your Merged ESP:
You should also re-open the merged ESP in xEdit after merging and use Check For Errors to make
sure everything is functional.


Removing Plugins from a Merged Plugin:
As of v1.7, you can remove plugins from a merged plugin. This will only work on plugins made with
Merge Plugins v1.7.

1. Load your merged plugin in TES5Edit.
2. Right click on the merged plugin, and click "Apply Script".
3. Choose "Merged Plugin Manager v1.0" from the dropdown menuand click OK.
4. Check the plugins you want to remove, then click "Remove".
5. Close the Merged Plugin Manager and then exit TES5Edit, saving the merged plugin.


Generating and Submitting Reports:
As of v1.7, you can generate reports on the mergabily of plugins. This can be done with a Merged
Plugin or with the ESP files you merged.

1. Load your merged plugin in TES5Edit.
2. Right click on the merged plugin, and click "Apply Script".
3. Choose "Merged Plugin Manager v1.0" from the dropdown menuand click OK.
4. Check the plugins you want to generate reports on, then click "Report".
5. Enter the information you want to submit for each report.  You won't be able to generate the
report if you leave one or more fields blank or at their default values.
6. Once you've generated your reports, go to your Edit Scripts folder and open the "mp" folder.
7. Run "Merge Plugins Report Client.jar", enter a username you want to have associated with
your report, and click Login.
8. Click "Send Reports" to send all reports you have generated to the remote server to be
incorporated with the Dictionary file.
9. Click "Logout".

NOTE:
If you're found to be abusing the report submission system your IP will be blacklisted and
you will no longer be able to submit reports or get updated dictionary files through the client.


Updating your Dictionary File:
Updating your dictionary file will give you all the latest definitions from reports submitted by
users.  I recommend you try to update your dictionary file before merging, and at most once a day.

1. Go to your Edit Scripts folder and open the "mp" folder.
2. Run "Merge Plugins Report Client.jar", enter a username, and click login.
3. Click "Get Dictionary".
4. Click "Logout".



Customization:
//=====================================================================================================
Right now there is one user variable you can set to adjust how the script runs. You can find this user
variable at line 30 in the "constants" section.

debug: Set this to true to make the script print more messages to the log when it runs.



Frequently Asked Questions:
//=====================================================================================================

Q: SkyRe? AutomaticVariants? SUM?
A: Don't touch plugins that were made or are in any way related to these mods. SkyRe was made
with TESVSnip, and is a giant mess of corrupted data. In addition, all third party patchers will create
corrupted data, which will lead to CTDs if copied in TES5Edit.

Q: Will <insert mod here> work?
A: Check the testing thread. If it's untested then there's only one way to find out... try it!
If things don't work post a report and restore your old ESPs.

Q: What mods don't merge well?
A: Mods which have invalid or out of order subrecord errors will cause CTDs. Other than this, the
greater majority of mods merge well.

Q: What are the benefits of merging?
A: Merging reduces ESP clutter allowing for more mods. It also makes your data directory
more manageable, so you no longer have to rip your hair out trying to find that ONE esp which is
causing you problems. It also improves performance, load times, and reduces CTDs. It's basically
the panacea for DMS: Disorganized Modder Syndrome.

Q: Why are faces grey?
A: This shouldn't happen anymore in v1.5. But if it does, you probably have to re-generate
face geometry for NPCs. Open the Creation Kit with your merged plugin as the active file, select
the NPCs in the merged plugin, and press CTRL+ALT+F4.

Q: EVERYTHING IS INVISIBLE OR PURPLY, MAN!
A: Extract the BSAs of the mods you merged. I talked about this in the "Tutorial" section.

Q: Right clicking isn't doing anything!
A: Wait for the background loader to finish loading the mods.

Q: I'm getting CTDs, why?
A: If a merge is giving you CTDs it usually means that one or more of the files you merged doesn't
like being merged. That is to say, something went wrong. You can try to isolate the problematic file
and not merge it by merging mods one at a time, which I highly recommend for everyone who doesn't
have experience merging plugin files.
OH and, if you didn't CHECK FOR ERRORS you may have merged mods with corrupted data that
TES5Edit can't read, resulting in CTDs.

Q: Will this make my game explode?
A: Just make sure you only save the merged ESP and disable the ESPs you merged from your
load order when testing the merged file. If things don't work for some reason restore your old
ESPs and delete the merged file and voila, everything's back to the way it was! No harm done.

Q: Where should I put the merged file in my load order?
A: At the spot where the highest mod you merged was/should be. So if you merge mods at
positions #10, #26, and #100 you want to put the merged ESP at position #10.

Q: Copy Intelligently? Copy Records? Copy Groups? What are these things!?
A: Just use copy intelligently. The other two are old merging methods that advanced users
can choose to use if they so wish. You don't need to worry about them.

Q: What is corrupted data and can I fix it?
A: Mods with invalid/out of order subrecords may have corrupted data. Or, at least, data
that TES5Edit can't interpret properly. Opening these mods in the Creation Kit as the active file
and then saving them MAY fix this problem, or it may not. If you merge mods with these errors
the result in Skyrim CTDing when it starts.

Q: My hair's gone?! My armor's gone?! My weapons are gone?! My face is gone?!?! WHAAATTTT?!!!?!
A: When you merge plugins all the records associated with them are being moved to new
FormIDs. That means all save games with references to the items from the mods you merged will
have the items/objects removed. This is just the way the game works, there's nothing I can do
about it. So say you merge your hair mods and you load a save game where you had a hair from
one of those mods. You will be bald. It's no biggie though, just use showracemenu, find the hair,
and save once you've fixed your depilation dilemma. Armors: add them back. Weapons/items: add
them back. Face: uhh... o_o'? Generally speaking, however, it's probably best to start on a new
save after merging hair/object/Leveled List/NPC outfit altering mods.

Q: Should I check for errors before merging?
A: Yes.

Q: Should I check for errors after merging?
A: Yes.

Q: Can I merge my entire load order into a single ESP?
A: Theoretically, if all those mods didn't have problems with merging, yeah you could . But
as of now there are several types of mods that aren't merging proper quite yet, so this isn't really
realistic. It'd take awhile and if you would have a problem you would have just wasted a lot of time.

Q: Can I merge merged files?
A: Yes, just not during the same run of TES5Edit (save and quit before each new merge).
This shouldn't introduce new problems, just remember to backup so if you merge something bad
you can undo it and keep the merged file you've been working on.

Q: Can I merge ESP patches into their master file?
A: This feature is being worked on, but it's rather complicated. I wouldn't try this unless you
really know what you're doing.

Q: Can I merge a mod's patches together?
A: Yes.

Q: Should I submit user reports?
A: Please do. It takes you all of a few minutes and may be very helpful to other users.

Q: I'm getting 'Overflow while converting variant of type (Int64) into type (Integer)', why?
A: You're opening too many mods in TES5Edit. Open only the mods you want to merge.
(exceed 128 mods in TES5Edit and this will happen when attempting to get formIDs from records in
mods whose load order > 128).

Q: What's with the file selection window?
A: If a file isn't in the dropdown list (yes, that's a dropdown list) you can't merge into it.
Choices other than -- CREATE NEW FILE -- will be added to the list if you're not using two-pass
copying and there are files loaded in TES5Edit at higher indexes then the highest index you have
selected. So if you're merging a file at the bottom of the list of loaded plugins in TES5Edit this list
won't have any options other than -- CREATE NEW FILE -- regardless of whether or not you
checked two-pass copying.

Q: I'm unable to select the Merge Plugins script from the Apply Script window/which script do I select?
A: When right click and choose apply script the window that comes up should have a
dropdown menu from which you must select Merge Plugins v1.7. DO NOT USE "Merge plugins",
this is a WIP non-functional script created by Zilav. If you do not see Merge Plugins v1.7 in the
window you're doing one of a few things wrong:
  • You're starting TES5Edit from a shortcut, which is starting it in a directory where it doesn't
  • have access to "Edit Scripts/Merge Plugins v1.7.pas". Try starting TES5Edit directly from
    the executable located in the directory as the Edit scripts folder you installed my Merge
    Plugins script into.
  • You didn't install my Merge Plugins script correctly. See the installation section.


Q: Can I remove an esp from a merged plugin?
A: Yes!  As of v1.7 you can do this with the Merged Plugin Manager!

Q: mteFunctions.pas is included in both AutomationTools and the Merge Plugins script, which should I use?
A: Use the one in the Merge Plugins script. If you really want to be certain you're using
the most up to date version (all versions are backwards compatible) you can grab that from
my GitHub.

Q: When should you renumber FormIDs and when shouldn't you?
A: You should renumber formIDs 99% of the time. The other 1% is when you're merging a file
that serves as a master for other files (this is extremely complicated and I can't really support you
on it, you'll have to figure it out yourself), or when you're merging files that have already had their
FormIDs renumbered by the Merge Plugins or Renumber FormIDs for Multiple Files script to
non-overlapping ranges.

Q: How do I use this with Mod Organizer?
A: It shouldn't matter where TES5Edit is installed if you're using Mod Organizer, as Mod
Organizer will make its virtual data directory available to TES5Edit so asset copying can function
correctly regardless of TES5Edit's starting directory.  Just make sure you're starting TES5Edit
through Mod Organizer!

Q: TES5Edit has to be in its own folder?
A: That's how I like to organize things. TES5Edit.exe just needs to be installed somewhere
in your Skyrim directory. It could be dropped in some obscure location in your DATA folder, in its
own folder, loose by TESV.exe, etc. So long as it is contained within your Skyrim directory, and
your Skyrim directory ends in the word Skyrim, things should work out fairly well as far as the
Merge Plugins script is concerned.

Q: In previous versions of this script certain plugins didn't merge well. Is this still the case?
A: Well, we now have a way of submitting reports and a dictionary file with definitions.  We'll
get a really good idea in the coming months just how many plugins don't merge well.

Q: Can mods with MCM menus be merged?
A: As of v1.7 MCM translation files are handled properly, so MCMs should merge perfectly. 
The only real issue that some mods with MCMs can have is issues of cross-mod scripting.

Q: Can I use SUM/AutomaticVariants/the Reproccer on a merged plugin?
A: Yes. A merged plugin is a plugin just like any other plugin is a plugin. A merged plugin is
just a plugin with data that used to be spread out between multiple plugins.

Q: When do merged plugins cause problems?
A: Merged plugins fail/cause problems when data isn't properly copied to them by TES5Edit
or assets aren't properly made available to them. Merged plugins are plugins just like all other
plugins are plugins.

Q: Can I merge ESM files?
A: If the ESM files are used by any other files in your load order you should not merge them.
If they aren't used by any other files in your load order, you can merge them. You could merge
ESMs that are used by other files in your load order if you uncheck Renumber formIDs and then
changed the master references in those files, but that's a bit difficult. Alternatively you could
open the master file and all the files that use it as a master, perform your merge with
renumbering formIDs, and then change the master references, but this is relatively untested.

Q: Can older merged plugins I made continue to work with new versions of TES5Edit/the script?
A: Yes. You don't need to re-merge things whenever there is an update to TES5Edit/the
merge plugins script. If the merge is working, it's working.  However, you won't be able
to use the features of the Merged Plugin manager on Merged Plugins that were made in
versions prior to 1.7.

Q: I'm getting grey faces still... why?
A: You need to extract your BSAs before merging.

Q: I got "Error in unit 'mteFuntions' on line 286..." why?
A: You installed an older version of mteFunctions.pas. Reinstall the latest version of the
merge plugins script and make sure you replace your current mteFunctions.pas with the one
in v1.7.

Q: How do merge plugins behave with BOSS/LOOT?
A: BOSS/LOOT don't know how to place merged plugins in your load order. You have to
manually place them at the highest load order position of the mods you merged with respect
to masters.  (so not above a file that is a master for the merged plugin)

Q: Is it OK to rename merged plugins after merging?
A: Yes. It shouldn't be required by any masters, so renaming it shouldn't affect anything.

Q: How much can this reduce my load order?
A: I've heard reports of ~70% plugin reduction in load orders (insaneplumber, 357 to 104
plugins). It all depends on the mods you're using.