I tried several methods to fix the bug but I had various glitches that I could not bring myself.
Finally I find a sort of solution when i see the mod: Closer Bolts - Closer and Hidden Crossbow Bolts from EvilDeadAsh34.
So I edited the skeleton.nif to create a new node for a special garment called "Bolt Quiver" in game and open the steelbolt.nif and dwarvenbolt.nif to remove the quiver and rework the arrow.
This mod hide the "real" bolt quiver and add a special garment that you can make at the forge in the misc section.
Now the quiver is closer to your body and don't go far away when you crouch.
Installation:
Requires Dawnguard
As usual, copy the content in your Data folder.
Incompatibilty:
Unfortunately this mod is incompatible with custom skeleton but this can be easily fixed with nifskope.