NOTE: Starfield Core, on its own, does nothing. It is meant to be a dependency for other mods. If a mod requires it, they will have it listed.

With the amount of content within Starfield, and the game being a lot more script-heavy than previous titles, the strains on Papyrus will be reached quicker as players expand out their mod lists. Script-heavy mods will be launched and further strain the system. In turn, many players will push false positives to mod authors where bugs and issues may simply be from overloading the game.

On top of this, trying to build out a script-heavy mod is difficult when attempting to both keep steady performance for the player, and to allow the mod to play friendly with the rest of the modding community. Between limitations within Papyrus such as 128 array element limits, script locking/unlocking, and 1.2ms time slices, problems are further compounded as authors build independent to one another. Many functions that in the past require a script extender further add complexities for someone using a mod, especially someone new to the entire process.

Enter Starfield Core. At its.....core are a handful of features that can allow all mod authors to use Starfield scripting to its fullest while expanding out the capabilities of their own mods, communicate down to the instance level within their mod and/or with other mods (whether installed or not), and overall just build a better experience for their users.

• Collections with 128 or 16384 generic element support: List (General), Queue (FIFO), and Stack (FILO)
• Debug logging system to allow consistent logs for all mods
• Listener system which allows scripts to register to other scripts and handle their messages as events
• Messaging system which allows script instances to register, send, and receive messages like function calls
• Parallel looping mechanism for threaded loops to improve performance
• Pool system which allows registering new objects dynamically as well as object instantiation and recycling
• Thread system which allows asynchronous tasks
  

Starfield Core supports List, Queue, and Stack with either default Papyrus 128 or Starfield Core "large array" flagged 16384 element limits. These are completely generic (via Papyrus Var type) though are not type safe (as there's no guarantees to force every Var in an array to be the same type). These collections are setup similar to other programming languages with typical functions and properties like Add(), Contains(), Clear(), and Remove().

List functions like traditional arrays and allow items to be added, inserted, or removed based on the item index or type. Normally can be used if a mod author needs direct access to all elements within the array. They can be a direct replacement for Papyrus arrays but allow data to be passed between the Var functionality used within Starfield Core as well as support the extend 16384 element limits.

Queue provides a FIFO (First-In/First-Out) array for scenarios that require it. Mod authors can enqueue items to the end of the array, and dequeue items from the start of the array. There is no access to items within other than the first element.

Stack provides a FILO (First-In/Last-Out) array for scenarios that require it. Mod authors can push and pop items to the top of the array. Similar to Queue, there is no access to items within other than the last element.

All collections are instantiated via the Pool System, and allow moving of arrays using the Var mechanism across all systems in Starfield Core. Collections can also be recycled within the Pool System when returned after use. However, they can cause memory leaks if their contents are also instantiated via the Pool System and are not returned, so a mod author should be aware when using them.


If (Array.Length < ArrayMax)
;{
List newList = PoolSystem.GetPoolObject("List", None) as List;
newList.Add(item);
Count += 1;
Array.Add(newList);
Return True;
;}
EndIf

Function CheckForInitialize()
;{
    If (PoolSystem.Initialized && ThreadSystem.Initialized)
    ;{
        _registeredMailboxes = PoolSystem.GetPoolObject("List", None) as List;
        _registeredMailboxes.LargeArray = true;
        _registeredMailboxes.Resize();
        _queuedMsgs = PoolSystem.GetPoolObject("Queue", None) as Queue;
        _queuedMsgs.LargeArray = true;
        MailboxId MailboxId = new MailboxId;
        _nextMailboxId = MailboxId.Dynamic - 1;
        _initialized = True;
        Self.StartTimer(_messagesUpdateLength, _messagesTimer);
        DebugExt.Log(3, "SQ_Dispatcher", "CheckForInitialize", "Dispatcher has initialized");
    ;}
    Else
    ;{
        Self.StartTimer(_checkForInitializeTimerLength, _checkForInitializeTimer);
    ;}
    EndIf
;}
EndFunction

Function SendMessage(Dispatch message)
;{
Guard MessageGuard
;{
_queuedMsgs.Enqueue(message);
;}
EndGuard
;}
EndFunction


Having a consistent approach to Papyrus debug logging between mods can help to quickly identify where issues are occurring, especially if mod authors request logs from users having issues. Starfield Core adds DebugExt.Log which requires passing in the type of log message (i.e. error, info, warning), the script name, the function, and a description. This keeps the log cleaner to read when scanning through it, and a mod author can quickly search for their own script files within logs to see if everything is working as intended.

Like a "try/catch" exception handler in other languages, the most efficient usage of DebugExt would involve mod authors being able to check conditions on all scripts as required to alleviate Papyrus itself having to handle the error. The "catch" portion would send a DebugExt.Log with the error type which can quickly be scanned within the log to determine what the problem is. As the focus in Starfield Core is performance, the less Papyrus at the engine level needs to handle exceptions (which could produce undefined results throughout the entire game), the better the experience is for all mods and users.

Log message types:

• Int Critical = 0 const;
• Int Error = 1 const;
• Int Warning = 2 const;
• Int Info = 3 const;
• Int Verbose = 4 const;
• Int Debug = 5 const;
  


DebugExt.Log(1, "SQ_Dispatcher", "OnTimer", "Messages cannot be enqueued to mailbox");
DebugExt.Log(2, "ExampleQuest", "MessageHandler", messageType + " not handled");
DebugExt.Log(3, "List", "Add", "Adding a new element to the list");


As requested by the community, the Listener system is an extension of Dispatcher that adds an event and intercept feature to Starfield Core. Scripts (with a registered mailbox) can listen to other scripts (also with a registered mailbox) and intercept their messages. This allows scripts to utilize Listeners as an event system (when the main script receives a message, the listeners immediately know it was received and can handle this as an event based on their individual needs), and/or an intercept system (a "parent" script could have many "child" scripts that have unique functionality when the parent changes, and these scripts can make those changes whenever the parent receives its messages).

The process below for the messaging system (called Dispatcher) applies for most of the Listener system in terms of registering mailboxes, sending messages, etc. The only major functions that defines this feature are AddListener (passing in the mailbox id a mod author wants to listen to, and the mailbox id of the listener) and RemoveListener (same parameters). This functionality does not impact the normal messaging system, with the original mailbox none the wiser that others are listening other than a new ref counter ensuring that all mailboxes (main and listeners) can properly check the message being sent before being recycled.


Function CheckForInitialize()
;{
If (ExampleQuest.Initialized)
;{
MailboxId mailboxId = new MailboxId;
_mailbox = Dispatcher.AddMailbox(-1);
_mailboxId = _mailbox.GetMailboxId();

Dispatcher.AddListener(mailboxId.Core, _mailboxId);

Self.StartTimer(_exampleUpdateLength, _exampleUpdateTimer);
_initialized = True;
DebugExt.Log(3, "ExampleListenerObject", "CheckForInitialize", "ExampleListenerObject has initialized");
;}
Else
;{
Self.StartTimer(_checkForInitializeTimerLength, _checkForInitializeTimer);
;}
EndIf
;}
EndFunction

Bool Function AddListener(Int mailboxId, Int listenerId)
;{
Guard MessageGuard
;{
If(_registeredMailboxes.ContainsAt(mailboxId))
;{
Mailbox mailbox = _registeredMailboxes.Get(mailboxId) as Mailbox;
mailbox.AddListener(listenerId);
Return True;
;}
EndIf

Return False;
;}
EndGuard
;}
EndFunction


Papyrus locks an entire script file when any worker thread enters it. While in most cases, this isn't a huge problem in smaller mods, in busier mods this slows processing down as many threads queue in to complete what they're doing. Issues can arise when external calls are made, and the same thread has to queue back in to complete its work which may or may not result in undefined behavior (especially true if another thread changes data before the original call completes).

Starfield Core adds a messaging system (called Dispatcher) which completely decouples all scripts from one another when used. Each script instance registers within the Dispatcher and is provided a Mailbox with a unique id (from 0-16383). Scripts can then make "function calls" to any other script (down to the instance level) via sending a message to that instance's mailbox id.

Messages (called Dispatches) are built with a sender (mailbox id), a recipient (mailbox id), a message type (string that acts like a function call), and Var[] for parameters. They are passed to the DispatchRouter, which then works with Dispatcher to get the message over to recipients as quickly as possible. Scripts then handle their own messages by querying their Mailbox and dequeuing messages to their custom message handler.

Why is this beneficial for mod authors? First, it alleviates the need to have all functions created and fleshed out prior to a call happening. This is especially true for intercommunication between mods. A mod author can send a message to another mod's script and, even if that mod is not installed, it simply does not handle/send the message.

Second, by decoupling all scripts, worker threads entering a script do not need to queue back in. This helps greatly with performance as threads can simply complete all operations within a script without being interrupted. For busier scripts, they simply receive more messages to handle and can use other systems like Threading to handle the increase in message throughput.

The end goal for Starfield Core is to have mailbox ids reserved as blocks by various mod authors utilizing it. These can be hardcoded into the mod and make it easier to communicate between all mods using Starfield Core as a dependency via Enum (our mod StarSim already reserves some ids for core systems). Also if/when the 16384 limit is hit, expanding to a higher amount would be straightforward.


Function HandleExampleMessage(Dispatch incomingMessage)
;{
    DebugExt.Log(3, "ExampleQuest", "HandleExampleMessage", "ExampleQuest received message");
    Dispatch exampleMessage = PoolSystem.GetPoolObject("Dispatch", None) as Dispatch;
    MailboxId mailboxId = new MailboxId;
    Var[] newMessageData = new Var[1];
    newMessageData[0] = Self as Var;
    exampleMessage.Create(mailboxId.Core, incomingMessage.Sender, "ExampleMessageResponse", newMessageData);
    DispatchRouter.SendMessage(exampleMessage);
    DebugExt.Log(3, "ExampleQuest", "HandleExampleMessage", "ExampleQuest sending response message");
;}
EndFunction

While ((_mailbox != None) && _mailbox.ContainsMessages())
;{
    DebugExt.Log(3, "ExampleObject", "OnTimer", "ExampleObject has messages to handle");
    MessageHandler(_mailbox.Dequeue());
;}
EndWhile

Function MessageHandler(Dispatch incomingMessage)
;{
    String messageType = incomingMessage.MessageType;

    If (messageType == "ExampleMessageResponse")
    ;{
        HandleExampleMessageResponse(incomingMessage);
    ;}
    Else
    ;{
        DebugExt.Log(2, "ExampleObject", "MessageHandler", messageType + " not handled");
    ;}
    EndIf

    PoolSystem.ReturnPoolObject("Dispatch", incomingMessage);
;}
EndFunction


For computationally heavy loops, these can bog down a single Papyrus thread running through them. The problem can further compound when there are external function calls within the loop causing the thread to have to queue back in multiple times. These are prime candidates for parallel loops, which Starfield Core provides support for.

With ThreadSystem.For, loops can be created and queued into the Thread System. These use generic thread functions, defined like function delegates in other languages, which are passed a Var[] for parameters. If the loop is entirely asynchronous, nothing more is necessary, while if the calling function needs to know when the loop is complete (or if it breaks early or is stopped), the ThreadSystem.For call returns a struct that can be checked against for IsCompleted.

Loops share a struct to allow communication between them, and can respect if any iteration has called Break or Stop (though it cannot guarantee that, based on the order threads pull the individual tasks in, that a particular iteration after a break has not already started). While Var cannot accept arrays, Starfield Core's collections can be passed and are the preferred method for sharing arrays between all features.

Note: There is overhead when creating parallel loops. Simply converting all loops to parallel will very likely result in a reduction of performance, especially in smaller loops with minimal computations. Like all code optimizations, mod authors should be profiling their scripts to see where something like a parallel loop could improve performance which will most likely be from computationally heavy code that makes many external calls.


Function ExampleCreateParallelFor()
;{
    DebugExt.Log(3, "ExampleQuest", "OnTimer", "ExampleQuest is creating parallel for");
    Var[] newTaskData = new Var[1];
    newTaskData[0] = Self as Var;
    _currentLoopState = ThreadSystem.For(0, 25, "ExampleQuest", "ExampleParallelFunc", newTaskData);
;}
EndFunction

Function ExampleParallelFunc(ThreadTask task) global
;{
    If (task.ParallelFor)
    ;{
        Int index = task.LoopIndex;
        SQ_ThreadSystem:ParallelLoopState loopState = task.LoopState;

        If(!loopState.Stop && (!loopState.Break ||
(loopState.Break && task.LoopIndex < loopState.LowestBreakIteration)))
        ;{
            Var[] params = task.Params;
            ExampleQuest testObject = params[0] as ExampleQuest;
            
 ; Loop body

            If (task.LoopIndex == 18)
            ;{
               loopState.Stop = true;
            ;}
            EndIf
        ;}
        EndIf

        loopState.LoopCounter += 1;
        If ((loopState.EndIndex - loopState.StartIndex) == loopState.LoopCounter)
        ;{
            loopState.IsCompleted = true;
        ;}
        EndIf
    ;}
    EndIf

    task.Completed = true;
;}
EndFunction


Instantiating new scripts is difficult as there's no direct New() call in Papyrus like in other languages (outside of arrays and structs). Various mods approach this differently such as creating new activators on the fly, "hardcoding" objects in holding cells, etc. but these are not always consistent between mods and rely on that particular mod to ensure these new objects are properly utilized and cleaned up when finished. The end result is mod authors can build mods quicker if they can simply instantiate the script they need on the fly, which Starfield Core provides via the Pool System.

Pool System allows new scripts to be registered and instantiated. Pool objects automatically register when a mod loads, and can then be instantiated by any other mod (with a type function to test if it exists via the unique string for each type). When a pool object has completed whatever purpose it has, it can then be returned to the Pool System to be recycled and used again.

Instantiating scripts (i.e. via PlaceAtMe() and similar functions), like a typical New() call in other languages, has a pretty high computational cost that should be avoided as much as possible. While there are recycling functions embedded into the engine to reuse forms as much as possible, that doesn't necessarily mean that a PlaceAtMe for an object does not have a cost if something has to be created on the fly. Pool System tries to exhaust its local pools of objects, based on type, before having to revert to creation of new objects.

PoolObjectType acts as the base script that all pool objects extend. These register automatically into Pool System after it fully initializes, and handle both the constructor and destructor of a script. Once a script is returned to any other script, it should be usable immediately without any additional setup required.


Function CheckForInitialize()
;{
    If (PoolSystem.Initialized && PoolSystem.IsPoolTypeRegistered("Thread"))
    ;{
        _loopTasks = PoolSystem.GetPoolObject("List", None) as List;
        _loopTasks.LargeArray = true;
        _threadTasks = PoolSystem.GetPoolObject("Queue", None) as Queue;
        _threadTasks.LargeArray = true;

        Int i = 0;
        While (i < MaxThreads)
        ;{
            DebugExt.Log(3, "SQ_ThreadSystem", "CheckForInitialize", "Creating thread " + i);
            _threads[i] = PoolSystem.GetPoolObject("Thread", None) as Thread;

            i += 1;
        ;}
        EndWhile

        _initialized = True;
        Self.StartTimer(_threadLength, _threadTimer);
        DebugExt.Log(3, "SQ_ThreadSystem", "CheckForInitialize", "Thread system has initialized");
    ;}
    Else
    ;{
        Self.StartTimer(_checkForInitializeTimerLength, _checkForInitializeTimer);
    ;}
    EndIf
;}
EndFunction

Var Function GetObject(Queue pooledQueue, Var[] params = None)
;{
Guard PoolObjectGuard
;{
If (pooledQueue.Count > 0)
;{
List pooledList = pooledQueue.Dequeue() as List;
pooledList.Enable(false);
pooledList.Initialize();
PoolSystem.DecrementPoolCounter();
Return pooledList as Var;
;}
EndIf
;}
EndGuard

List newList = PoolSystem.PoolMarker.PlaceAtMe(BaseList, 1, True, False, False, None, None, False) as List;
newList.Enable(false);
newList.Initialize();
PoolSystem.IncrementObjectCounter();
Return newList as Var;
;}
EndFunction


Thread System coordinates a series of worker threads to work through generic ThreadTask objects. Any script can pass in a ThreadTask built from a generic function and a Var[] of parameters. These are asynchronous and completed in no particular order based on how full the queue is currently (even though a task is later in the queue, it may complete before something more computationally heavy so care should be taken if anything needs to be executed in a specific order). Calling functions only receive back a struct to tell it whether the ThreadTask has completed its work if necessary.

For a mod that has work it needs to complete that is computationally heavy and doesn't need to be in order, ThreadTasks can allow a lot of work to be accomplished without overloading Papyrus. Thread System works in tandem with messaging to have the least amount of Papyrus threads necessary to efficiently complete as much work as possible across any load order amount. The more these systems are utilized, the less likely that Papyrus gets bogged down and causes script lag which can quickly impact the entire game.


Function ExampleCreateThreadTask()
;{
    DebugExt.Log(3, "ExampleObject", "OnTimer", "ExampleObject is creating new task");
    Var[] newTaskData = new Var[1];
    newTaskData[0] = Self as Var;
    _currentTaskState = ThreadSystem.CreateTask("ExampleObject", "ExampleTaskFunc", newTaskData);
;}
EndFunction

Function ExampleTaskFunc(ThreadTask task) global
;{
    Var[] params = task.Params;
    SQ_ThreadSystem:TaskState taskState = task.TaskState;

    ; Function body to handle, can access anything passed in via task.Params above
    ExampleObject testObject = params[0] as ExampleObject;
    DebugExt.Log(3, "ExampleObject", "ExampleTaskFunc", "Handling task from: " + testObject);

    taskState.IsCompleted = true;
    task.Completed = true;
;}
EndFunction


These example use cases are marked based on how proficient someone may need to be within Papyrus to use. In some cases, an advanced Papyrus coder can build similar features such as the generic collections, threading, etc. There are still benefits within Starfield Core such as how it saves an accomplished coder time in not having to reinvent the wheel, the increased performance from mods having efficient systems to utilize of which the more mods using these the better the performance gains, and especially in the intermod communication benefits the Starfield Core messaging system provides for the entire ecosystem.

• [All] Starfield Core adds a messaging system that allows instance level access to scripts, increased performance by only needing one Papyrus thread executing a script at any given time due to decoupling (communication between scripts is via async messages), intermod communication (whether installed or not, messages can be sent without having to test if a mod is loaded), and a unified way for the Starfield modding community to ensure that communication between mods is easy and powerful. StarSim already uses messaging directly to communicate between systems as well as every ship and station. The reserved systems required to run the simulation are already added into the messaging system's enum for quick access by StarSim and any other mod to communicate. Mods utilizing Starfield Core for the messaging system can also request blocks of mailbox ids which are then hard coded into the mod for quick access to check if that mod is installed in a user's load list and to send/receive messages from it.
• [All] Starfield Core adds parallel loops and thread tasks which run through managed threads to ensure performance isn't impacted by too many operations at any given time. While building the functionality for parallel loops and thread tasks is doable by advanced Papyrus coders, without a unified thread pool that can be tweaked based on how much of the modding community is using Starfield Core, these changes would be unique to an author's mod only and compound performance the more script heavy mods are in a user's load list each doing their own threads. As the usage of Starfield Core increases, systems like load balancing can be added for threads to more efficiently use shared Papyrus resources.
• [All] Starfield Core provides a simplified debug logging system to make it easier for all mod authors to pinpoint information about their mod (a simple search for a script name finds all instances in the log), and to help when users send back logs if a mod is not working correctly. It is designed so that reading through a busy log is cleaner and quickly tells you information such as what type of debug message (like an error or information), which script is sending it, which function is sending it, and a clear description. While there's nothing fancy in this versus any other usage of Debug.Log, many competing logging methods make trying to dig through the Papyrus log difficult (especially with errors Papyrus itself is throwing in between).
• [All] Starfield Core provides a unified pool system for all objects, including new scripts registered automatically via the system. The ability to instantiate on the fly is a major benefit for beginner and intermediate coders to quickly access scripts without learning methods to load scripts at runtime. More importantly, the ability to recycle when completed is a benefit to all mod authors as it ensures that other mods using resources aren't creating memory leaks from created objects that are never cleaned up.
• [Beginner/Intermediate] Starfield Core provides generic collections (List, Queue, and Stack) allow mod authors to have more powerful arrays. And with the large array flag, this allows authors to get beyond the 128 element limit in cases where their mod needs to hold more data. These collections are especially useful as all systems within Starfield Core use Var which cannot accept Papyrus arrays and must have collections sent if an array is needed. Even when not using Starfield Core beyond collections, built-in Papyrus functionality uses Var arrays as a way to pass parameters (such as custom events), and these collections can be used as an array replacement.
  

NOTE: Starfield Core, on its own, does nothing. It is meant to be a dependency for other mods. If a mod requires it, they will have it listed.

• Follow the same instructions as you prefer from the mod requiring Starfield-Core.
• Mod manager (Vortex/Mod Organizer) is preferred.
  

• Starfield.esm
• [Official DLCs]
• [Official Overlays or ESL plugins]
• Starfield-Core.esm (to allow StarfieldCommunityPatch.esm to use functionality as appropriate for fixes)
• StarfieldCommunityPatch.esm
• [Other mods]