the Sim Settlements forums!

Register a free account today to become a member! Once signed in, you'll be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!

IDEK's Logistics Station scripts running amok

ThunderboltDragon

Well-Known Member
Patreon Supporter
Messages
262
My active scrip situation is really really bad and the script issue is very likely to
be caused IDEK's Logistics Station, possible in conjunction with the FO4 DLCs.

Lately my game have been acting really weird. IDEK logistics station not working at
Far Harbour settlements + a few other settlement where a LogStn have been built after
purchasing+installing all FO4 DL, Game crashing (freeze crash) when fast travelling to
settlements (but not to non settlement locations) and also for some settlements crashing
when trying to enter workshop mode. Walking around the map and into settlements
worked just fine though. The settlers also did what they were assigned to do,
slept at their assigned plots and etc.

So I have been looking at the saves with Fallrim tools and it wasn't a pretty sight.
These are some sample savefiles:

FALLOUT4 SAVEFILE (last one before DLCs got installed)
Jessie the level 60 Human male, in Commonwealth on 4d.15h.45m.4 days.15 hours.45 minutes
(616/5241 xp). Save number 248, created on Thu Aug 23 20:11:05 CEST 2018.
8 active scripts

FALLOUT4 SAVEFILE
Jessie the level 70 Human male, in Commonwealth on 5d.7h.39m.5 days.7 hours.39 minutes (26/5401 xp). Save number 306, created on Wed Oct 10 20:36:53 CEST 2018.
8 active scripts. This is where the log. station scripts (4 of them) starts to appear
and from this point they increase by time.

FALLOUT4 SAVEFILE
Jessie the level 70 Human male, in Commonwealth on 5d.11h.10m.5 days.11 hours.10 minutes
(645/6020 xp). Save number 312, created on Fri Oct 12 09:30:18 CEST 2018.
26 active scripts (18 IDEK scripts)

FALLOUT4 SAVEFILE
Jessie the level 70 Human male, in Nuka-World on 5d.12h.18m.5 days.12 hours.18 minutes
(5168/10543 xp). Save number 315, created on Fri Oct 12 11:47:37 CEST 2018.
40 active scripts (35 IDEK scripts)

FALLOUT4 SAVEFILE
Jessie the level 79 Human male, in Commonwealth on 6d.18h.26m.6 days.18 hours.26 minutes
(5409/11459 xp). Save number 436, created on Sun Oct 28 14:39:20 CET 2018.
108 active scripts (100 IDEK scripts).

Earlier tonight I also went to Vault 111 for about 15 minutes but it didn't do any difference.

Then I decided to see what happened if I unchecked IDEK logstn in the plugin list in Nexus MM.
Everything seemed to work fine then. I could fast travel to settlements, I could walk
around with no problem, I could access workshop mode, I also upgraded
Jamaica Plain to L3 (16 settlers) with no problem (it crashed when I tried earlier today).

This is Fallrim save film analyse (without IDEK LgStn)

FALLOUT4 SAVEFILE
Jessie the level 79 Human male, in Commonwealth on 6d.19h.20m.6 days.19 hours.20 minutes
(5816/11866 xp). Save number 441, created on Sun Oct 28 18:25:09 CET 2018.
47 unattached instances 8 active scripts.

Does anyone have a clue why IDEK logistics station suddenly starts to spam scripts?
Related to Far harbour/routing problem? Mod conflict? Load order problem (sorted with Loot).
And is there any way to solve this (without starting from scratch)?

Also sorry the very long text.
 
Load order i don't think i had IDEK in the past at the Bottom of my LO and now at the top right after the masters.Im running 200 Mods and most of them are real heavy but i don't have Problems.Have you switch the Format to esl from esp?That can cause problems if you do it in a running game.Try to disable the Mod and uninstall it full from your Profile and then reinstall it Fresh maybe some other Mod have change something.
 
Load order i don't think i had IDEK in the past at the Bottom of my LO and now at the top right after the masters.Im running 200 Mods and most of them are real heavy but i don't have Problems.Have you switch the Format to esl from esp?That can cause problems if you do it in a running game.Try to disable the Mod and uninstall it full from your Profile and then reinstall it Fresh maybe some other Mod have change something.

I might have changed to .esl (have .esl version now) but wouldn't that affect all
logistics stations and wouldn't it appear earlier? I last updated it on june 9th so
that's 2,5 months + another week or two with far harbour.

The older (pre-DLC stations) still have a provisioner assigned to them
while the newly built stations have a settler assigned to them but the
settler doesn't become a provisioner.

I also get this error message when viewing the desktop in the log station:
KHstTEk.jpg


A working station from my current game:
z5gRtOY.jpg


*edit*
I have re-installed the mod but it didn't do any change.
 
Last edited:
The Error you can Fix most of the time with a Plot refresh.After wait some minute and try again to use the Terminal.To be sure that all real Work refresh them all.Bugs can happen very late sometimes.You can Play 100 Hour's and at Hour 101 the game get Buggy why the game ask for something that you cancel or change at the first Hour only at that moment as until that point was no need to ask for that.But even i see at the moment a lot of Bugs with many Mods they all seams to come out of nothing.Maybe its the Version Update from the Core game as i have the Feeling after every Game Update we get more Bugs and Mods don't work any more right.
 
If you could send me your save I could at least take a look at what it's doing, I've never had anyone have those sort of problems with my scripts before. ILS has the potential to spin up a ton of simultaneous Papyrus threads, but none of them should ever run for more than a few seconds.

Chances are that if scripts are getting hung up, some Papyrus thread somewhere went haywire and ILS's global "edit lock" got stuck on. I can only guess what might have caused this, but you might be able to rescue the save by just running:
Code:
cqf ILS_LogisticsParent ClearEditLock
I'd still like it I could track down what caused that to happen though, some of ILS's thread safety logic gets pretty convoluted and it's possible there's some super rare case that my scripts are handling wrong or something. I've never heard of this happening in any version of ILS I've ever released, but it's definitely happened to me in development versions.
 
Last edited:
@GladiusVictorius I have tried to refresh some of the none working ones
without any improvement.
@EyeDeck I'll send you a save tomorrow. I think I can keep the scripts
under control with fallrim tools. I can just delete the IDEK scripts which
won't solve the original problem but I can still continue on the save.
 
I recommend trying the command I posted first. ILS's scripts are structured so that any thread prematurely terminating has a pretty good chance of causing a deadlock that will never clear without running that console command.
 
What EyeDeck is referring to is something you're going to see more and more of in the community, and ultimately it's going to mean that we're going to have to try and end the community practice of terminating scripts in Fallrim Tools.

For example, many scripts in Sim Settlements, Workshop Framework, and Workshop Plus all have the same thing going on - if you terminate a script from one of them, you're very likely to cause all of the scripts in that mod to lock up until you run a particular console command.

Ultimately, with my mods, I'll be adding some sort of in-game thread clearing tool, whether it be a holotape or some other mechanism so that Fallrim will no longer be necessary. In the meantime, best to report issues like you've done, I think most of us who code with edit locks have a console command you can use to free them up.
 
Ultimately, with my mods, I'll be adding some sort of in-game thread clearing tool, whether it be a holotape or some other mechanism so that Fallrim will no longer be necessary.
Yeah, it occurred to me that it would be a good idea for me to put a manual [Clear Script Lock] button into ILS's menu somewhere. I have never had anyone report an issue with this before, but if for any reason ILS's edit lock variable gets stuck on, the whole mod will completely lock up until something explicitly unlocks it. It shouldn't be possible for this to happen in-game since Papyrus scripts are really tenacious once they get started, but if any of ILS's scripts are ever manually killed with external tools it's almost guaranteed that this will happen, and my mod currently has no provision for dealing with this without console commands.

In case anyone wonders why a script would ever be written this way, it's because ILS (and SS, the vanilla workshop system etc) are full of data stores that could get modified at any time. So, when some thread starts up that accesses or modifies one of these data stores, it runs a function I called GetEditLock() that either toggles on the edit lock variable if it's currently off, or waits for the edit lock variable to become clear. This ensures that the data that the "active" thread is operating on doesn't suddenly change because some other thread decided to modify it. Then, when that thread is done, it runs ClearEditLock(), and if there are other threads waiting in the queue then one of the other ones will jump to the front, do whatever it was going to do and so on. You can guess what would happen if some thread gets killed before it gets to run ClearEditLock()—every thread waiting in the queue will get stuck forever, and the queue will just get longer and longer until the game probably crashes at some point.

It's also entirely possible if I'm careless about my thread locks for ILS to deadlock itself. If I had a function that just, say, called GetEditLock() twice in a row, that would break the mod. It's easy enough to avoid doing that, but there are also places where, say, a thread starts up that needs an edit lock, which calls some other function that doesn't need an edit lock, which then calls a third function that does need an edit lock—and if I'm not careful, it's possible for that last function to try to get an edit lock, unaware that the thread it's running in has already gotten a lock, and oops, everything breaks. I'm reasonably sure I squished every instance where this can happen naturally, but ILS's scripts are several thousand lines long, so it's never certain.

I forgot to mention this before, but I assume that if the OP looked at his IDEKsLogisticsStation.0.log file (assuming logging is turned on), he'd see tons of lines like
Code:
A thread is waiting in WaitEditLock(). Queue count: 100
In normal operation it's very very rare for the queue count to get above 1 or 2. ILS's logs are crazy verbose, I just checked the number of separate places where lsTrace(), my log output function, is called throughout all of ILS's scripts and apparently that number is 346, so it's usually pretty easy to diagnose problems when they occur—though it helps if I remember to mention it.
 
I recommend trying the command I posted first. ILS's scripts are structured so that any thread prematurely terminating has a pretty good chance of causing a deadlock that will never clear without running that console command.

I tried the console command and it did clear the error message in the
Logistics station terminal. However, it still doesn't make the assigned settler
a provisioner and it still doesn't add the station to the network.
I refreshed one existing station with no change and also built a new SS-settlement
at Greentop Nursery where I put down an ILS.

I also had a look at the latest save with Fallrim tools and no active ILS script in it.

Obviously I am aware that messing around with deleting scripts with Fallrim Tools
might break stuff permanently but if the game would become unplayable anyway
due to script overload I felt that there weren't much to loose.
 
Yeah, it was definitely the spin lock getting stuck. The reason the mod isn't working after clearing the spin lock has to do with some additional thread safety code; basically, ILS has a few functions that have a tendency to (try to) queue up in the main spin lock several times in a row, for example the big "recalculate all supply lines" function would often queue up like four times in a row because something happened somewhere that necessitated that function to run. To prevent this, for a few functions I added a simple "is already in queue" bool, and any time those functions are called they would check if that bool is already set, and if so just bail out, since actually running those functions repeatedly wouldn't do anything useful. In your saves, some of those bools are set true, and that means that those functions think they're already queued up because the relevant thread started at some point, but evidently got killed before it finished and was able to clean up after itself, so they'll never run again.

To fix the save you sent me, you can do this:
  • Open your save in FallrimTools
  • Search for idek:ils:logisticsparentscript under "Script Instances"
  • Find RecalcResourcesQueued in the Data panel and set it to false
  • Do the same with RecalcSupplyLinesQueued
  • Set spawnedThreads to 0 (this is another variable that's set wrong because some thread died prematurely, though this one is less catastrophic than the others)
  • Save etc
If I remember my scripts well enough, everything should go back to working properly after this.

Whenever I get around to the next version, I'll put in a tool that can fix these sorts of things on demand. This is a pretty good example of why not to kill random scripts though, you can get scripts into strange, normally impossible states that only the relevant mod author is likely to know how to fix, and without their help the only way to get things working again is often going to be to do a "clean" install of the mod.
 
Last edited:
Hey @EyeDeck , sorry for the late reply. I have tried your suggestion
and it seems to work fine in the two settlements I have checked so far.
One of them was previously not-working and the other is a completely new.
 
Top