I added some shack foundations to the castle to repair the damaged walls. that one corner may not be pretty, but it gets the job done.

You'll need Transfer Settlements - Shareable Settlement Blueprints
http://www.nexusmods.com/fallout4/mods/22442