Crash on Castle City-wide improvement

[Skojanza]

I have no clue if its on my part. And I can only guess that SS really is the problem, as I don't see the message pop, but the castle is on 100% and I slept of 24 hrs IN the settlement after it reached that (there is also the fact that it doesn't crash when do the idiot thing of loading the save without SS).
I checked the trouble mods list and the only one I have installed is conquest, but it's not a conquest settlement, neither is it anywhere near one.
I will attach a copy of my loadorder. (would have linked a pastbin but rules.) ((also ignore that sim settlement appears disabled, I tried different stuff.))
No, it has not happend before.
Yes, I am using the latest version (the far harbor patch)
No, other settlements don't cause a CTD. (At least the majority of them, the ones I saw upgrade today with this load order)
Yes, it does happen anywhere. I have a short like.. 10 secs at max where I can do stuff, I tried fast traveling away, I even used coc to get into an interior, but it crashed either way.

 

[Skojanza]

So, I screwed around a little more, disabled every other mod. It still crashes.The only mods activated are the CC mods and Sim settlement. It still crashed, I also tried downgrading Sim Settlement, didn't change it. I also tried removing RotC and IR, it crashed. Only when I disabled the Sim settlement ESM did it stop.
I have the save file uploaded to dropbox. But I can't post the link. ((Should be one where only Sim settlement is installed and the RotC tables are gone, maybe?))
I guess that invalidates my assumption on it being the fact that it wants to upgrade.