CDC_PapyrusSpeedTester 1.0

by CDCooley
2012-06-15

for
The Elder Scrolls V: Skyrim

*********************************************************************

WARNING!!!!!! This is only to test Skyrim's Papyrus script speed.

On loading the game your character will be forced to an empty and
dark cell, scripts will run, and then the game will exit by itself.

If you're not a script writer worried about performance in your mod
then you should NOT be using this.


*********************************************************************

INSTALLATION, UPGRADE, and INCOMPATIBILITIES

If you need instructions for this, you should NOT be using it.

Wondering about upgrading? Keep reading because you shouldn't.

There should be no incompatibilities in the traditional sense, but
for best results you don't want any script heavy mods active.

SKSE can be found at http://obse.silverlock.org/ and is needed only
if you want to compare the speed of it's functions.


*********************************************************************

RUNNING THE TESTER

1) You must first enable Papyrus tracing and logging. but putting
these lines in your Skyrim.ini (in your My Games/Skyrim or
Saved Games/Skyrim folder). Add them or change them.

[Papyrus]
bEnableLogging=1
bEnableTrace=1

2) Enable the mod and load any saved game where your character isn't
currently doing something active (but you are in control). Don't
start a new game with this active because the scripted introduction
sequence messes up the timers.

3) If everything is working correctly you should see a sequence of
startup messages and then the tests will run. (If the sequence
stops, just reload the game. The scripts fail to fully activate
occasionally.)

4) Results of the tests will be shown on screen, but they are also
being recorded in the log file so don't worry about trying to
remember them or copy them down. The on-screen notices are just
there so you can see that the tester is still making progress.
(And maybe to stop early after you've added your own tests.)

5) Ten minutes or more later when the tests are complete, you'll get
two final messages and then the game will quit automatically.
Tests are run in the newest-to-oldest order, so after you've
seen the notifications appear on screen for all of your new
test cases you can just quit the game and go check to log to
see those results. You don't have to wait if you're impatient.

6) Look at the log file for the list of results. There should be
a Logs folder in the same location as your Skyrim.ini file.
Inside of it there is a Script folder, and the log file itself
is inside that with the name Papyrus.0.log. If you run the tests
multiple times, you'll get more numbered log files, but the most
recent one will always be Papyrus.0.log.

7) Disable the mod when you're done testing because the game is
unplayable with it installed!


*********************************************************************

ADDING YOUR OWN TEST CASES

The main script that runs the tests is CDC_PapyrusSpeedTester and
it is attached through a quest alias to the player. The secondary
script CDC_PapyrusSpeedTesterContainerOR is an ObjectReference
script attached to a container in the CDC_PapyrusSpeedTestCell.

At the top of both are some instructions, but the process is simple.

Each test case consists of two functions in a state. Copy one of
the existing TestCase states and change it's number to be one
higher than any of the existing cases. (They are in the file in
reverse order to make this easier.) Update the "NumberOfTestCases"
variable to your new case number. Edit the new test case's code
and description functions.

There's a TestCaseX dummy state with comments to show you were
you can make your changes and 45 examples already provided.

You can declare new properties and variables at the top of
the script but don't modify the ones already there unless you
are sure you know what you are doing.

You can also modify and delete some of the existing test cases if
you prefer not to run all of them. Of course your new test cases
will always run first, so you can also just quit the game early
after the test cases you care about have already run.

You can add tests to either of the existing scripts or use the
container version as a model for another of your own. The link
is to add a property for your new object that will hold your new
script and then add a call to get things started in the OnInit
event of the main script.


*********************************************************************

READING THE RESULTS

Here's a sample of the results on my machine. The first two are
just show how much time it takes just to run the testing framework
itself. The other results are adjusted based on these so you can
see how long it takes to run just the test commands themselves.

Results will vary from one run to the next, so don't be fooled by
too many decimal places of the values. With multiple runs it's clear
that the GetVersionNumber and GetVersionRelease functions are really
the same speed. They both are about 12.67 on my machine. The
GetBaseObject function takes about the same time, but the results of
test 43 are always a little higher because the "if" statement and the
"as Potion" cast also take a small amount of time.

Look through the list and you'll probably be surprised by just how
much difference there can be between two or three ways of doing
the same things.

s, f, i, and b are simple string, float, int, and bool variables.
ref and other are ObjectReference variables.
myActor is an Actor variable.
PlayerRef and GameDaysPassed are both properties.

And since line breaks would mess up my listing, they are shown
with the | character.

Tests 27 through 22 show that accessing a value from a local
variable or property is much faster than calling the functions
to get that value and Game.GetPlayer() is by far the slowest
method.

Notice that the smallest amount of time to get something useful
done (i = 5) is about 0.008734 and the slowest functions
(GetPlayer, GetDistance, etc.) are over 1000 times slower!

And unfortunately some functions (GetDistance, GetPositionX,
and even IsLocked just appear to be naturally slower) than
other operations.

One other thing I've found is that those functions don't get
noticably slower when my FPS drop to around 30 from 60 but the
faster functions are affected. So I believe the speed difference
is related to them being external calls that need to switch
context in the scripting engine. Maybe if the ObjectReference
were attached to the reference instead of the base container
object things would be faster but I doubt it and didn't test.


Papyrus Speed Tester
ObjectReference script tests
Overhead Time: 0.013600 used by the timing function itself
Baseline Time: 0.013199 an 'empty' loop 1000 times
Test case 8: 12.858016 f = GetDistance(PlayerRef)
Test case 7: 12.673988 f = self.GetDistance(PlayerRef)
Test case 6: 12.948014 f = PlayerRef.GetDistance(self)
Test case 5: 12.888017 b = IsLocked()
Test case 4: 12.753998 f = GetPositionX()
Test case 3: 12.745988 f = self.GetPositionX()
Test case 2: 13.131989 f = X
Test case 1: 13.151978 f = self.X
ReferenceAlias script tests
Overhead Time: 0.013100 time used by the timing function itself
Baseline Time: 0.013400 'Empty' Loop 1000 times
Test case 45: 12.648018 Debug.GetVersionNumber()
Test case 44: 12.712014 SKSE.GetVersionRelease()
Test case 43: 13.050011 if ref.GetBaseObject() as Potion | endif
Test case 42: 12.551986 if ref.GetType() == 46 | endif
Test case 41: 0.012098 if GameDaysPassed as GlobalVariable | endif
Test case 40: 12.718003 if GameDaysPassed.GetType() == 9 | endif
Test case 39: 0.009546 s = "sample"
Test case 38: 0.029388 i = GameDaysPassed.Value as int
Test case 37: 0.027518 f = GameDaysPassed.Value
Test case 36: 0.028434 i = GameDaysPassed.GetValueInt()
Test case 35: 0.017072 f = GameDaysPassed.GetValue()
Test case 34: 12.950012 f = Utility.GetCurrentGameTime()
Test case 33: 12.739990 f = PlayerRef.GetPositionX()
Test case 32: 12.797974 f = ref.GetPositionX()
Test case 31: 12.589997 f = PlayerRef.GetPositionX()
Test case 30: 13.029968 f = PlayerRef.X
Test case 29: 13.028077 f = ref.X
Test case 28: 12.406006 f = PlayerRef.GetDistance(ref)
Test case 27: 12.921876 ref = Game.GetPlayer()
Test case 26: 0.031696 ref = GetRef()
Test case 25: 0.021184 ref = GetReference()
Test case 24: 0.009194 ref = PlayerRef
Test case 23: 0.022252 ref = Game.GetForm(0x14) as ObjectReference
Test case 22: 0.009192 ref = other
Test case 21: 0.012604 myActor = other as Actor
Test case 20: 0.010586 ref = None
Test case 19: 0.012588 i = 1 | if true | i = 2 | endif
Test case 18: 0.008744 i = 1 + (true as int)
Test case 17: 0.015898 i = Math.Floor(0.04)
Test case 16: 0.008842 i = 0.04 as int
Test case 15: 0.014084 if b | i=1 | endif | if !b | i=2 | endif
Test case 14: 0.009972 if !b | i=1 | else | i=2 | endif
Test case 13: 0.014116 if b | i=1 | elseif !b | i=2 | endif
Test case 12: 0.012662 if b | b=false | elseif !b | b=true | endif
Test case 11: 0.012398 if b==true | b=false | else | b=true | endif
Test case 10: 0.010344 b = !b
Test case 9: 0.008732 int n = 5
Test case 8: 0.008734 i = 5
Test case 7: 0.010072 if !i | i = 5 | endif
Test case 6: 0.011458 if i > 0 | endif
Test case 5: 0.010316 if i == 0 | endif
Test case 4: 0.010160 if !i | endif
Test case 3: 0.010128 if !b | b=true | endif
Test case 2: 0.008736 b = true
Test case 1: 0.007236 0
Testing Complete



*********************************************************************

FUTURE DEVELOPMENT & WISH LIST

More test cases?

More people running them and sharing the results.



RELEASE HISTORY

1.0 - 2012-06-15 - Initial public testing release.


*********************************************************************


CREDITS AND LICENSING

Thanks Bethesda and JustinOther.

This script is my attempt to improve on the method JustinOther has
posted on the Bethesda forums. These calculate and then subtract the
overhead of the testing framework itself so that the results should
be just the cost of the functions and operations being tested.

I also like to keep my tests around so I made the framework so I
can easily just keep adding more rather than changing existing tests.


Please feel free to extend and improve them. Then share them!


Please post comments and complaints on the Bethesda or NexusMods forums.

Charles Cooley
[email protected]
cdcooley on the NexusMods and Bethesda forums
2012-06-15