0 of 0
Documentation
Readme
View as plain text
___________________________________________
- PSSG TOOLs -
And Modding Tips
[GreedFall]
___________________________________________
May 22, 2020
Version: 2.04
Written by: Dheu
Email: [email protected]
Use subject: PSSG TOOLs 2.04
QUICK SUMMARY
-------------------------------------------------------------------------------
A collection of PSSG tools I created while modding GreedFall. Internet searches
for PSSG don't provide many hits, so I thought it best to put this out there.
This distribution also includes modding tips and advice/examples for developers
looking to mod GreedFall. It is by no means comprehensive. Just some organized
notes I made while working on my mods.
-------------------------------------------------------------------------------
I. > > > > PSSG Overview
-------------------------------------------------------------------------------
What is PSSG? PSSG is a container format for game resources. Basically it is
a custom Binary XML representation.
Imagine you have an OGG audio file. In the Silk Engine, it would save the
resource in a .pssg file. It "wraps" the ogg file with binary data that the
game needs to use the data within the game. These wrappers have a header
that indicate the contents to the engine and then a body with the data
itself. Sometimes a single PSSG wraps multiple resources.
If you know the PSSG header format, you can figure out where the ogg data is
and extract it.
Some PSSG files wrap other binary data in whole. For example, the audio
data is a fully independent ogg audio file. So if you extract the HEX from
the PSSG and throw it into its own file, you can play it.
Other types such as DDS images will have the raw image data, but will excluded
the DDS headers since the PSSG container already has the information provided
by a DDS header. In these cases, you may have to add a header to the binary
content in order to view in a regular viewer.
History and Compatibility:
--------------------------
From what I can tell, PSSG was a format invented for the Phyre Engine: A Sony
PlayStation engine released in 2008 to promote creation of games for the PS3.
The engine and dev environment were made free because Sony wanted games for
their platforms. As far as I can tell, support for the engine ended in 2016
when the PS4 came out. However, it was free and cross-platform. So a number
of development shops took the engine and expanded it into their own
customized versions (updating it in-house to meet the demands of newer
Direct X standards). The Ego Engine and more recently the Silk Engine are
byproducts of the Phyre Engine. GreedFall uses the Silk Engine.
So why the history? These dev shops didn't customize everything. Only the
parts that they needed. So an app made for an ego engine game will often
times work with silk engine pssg files.
EGO PSSG Editor
---------------
One such program is the PSSG Editor created by Ryder25 mentioned in the
Standard Tools section of the GreedFall modding tips. The latest release of
EGO PSSG Editor was 2017. The EGO PSSG EDITOR can edit most PSSG files and
can be used to update many resources, however PSSG has eveolved. Silk engine
in particulat made changes to the spec that PSSG EDITOR can't handle.
Sometimes you try to open a PSSG file and the editor just barfs.
More concerning is that I have found situations where it silently fails
(doesn't inform you of any issues) and corrupts the output. An example is
the music/audio files in greedfall. If you convert a PSSG file that
wraps audio to XML using EGO and then converted it back to PSSG with no
changes, sometimes the PSSG is a different size. And if you place that pssg
in the overrides directory (as a pgz), the associated audio in game may
stop working.
PSSG CONVERTER
--------------
To help with some of the issues mentioned above, I made pssgConverter.py. It
took about 6 weeks of poking and prodding to figure out the Silk PSSG format
and then another 6 weeks to get a stable script. When I repeat the steps
that broke the audio using the EGO PSSG EDITOR with pssgConverter,
everything works as expected.
The key to the script is that it is not schema driven. It reads the file in
and makes a best guess at the schema for that specific file. It then saves
the schema to a .pschema file and writes out the data to XML. Users can then
update/edit the XML file and convert the XML back to PSSG. They can also
update the .pschema file to render some of the binary chunks in different
ways to make editing the XML easier.
PSSG CONVERTER is a command line script, so it isn't as nice or user friendly
as EGO PSSG Editor, but as a command line script, you can also easily run it
on many files at once. My GreedFallModding tips make use of this ability to
extract all the VOCAL files to XML so that the spoken line (part of the LIP
data) can then be extracted from the XML using a script to create a list of
all lines spoken by any given NPC. With thousands of lines, doing something
like that with EGO PSSG Editor would be impracticable.
-------------------------------------------------------------------------------
I. > > > > PSSG CONVERTER
-------------------------------------------------------------------------------
A. Overview:
pssgConverter.py understands PSSG. It converts between PSSG formats and
XML. The XML can then be edited and converted back to the original
resource for use in game as an override.
B. Prerequisites:
You will need 7-zip to extract the .spk files to a directory outside
greedfall and expose the games resources.
You will need to install Python. pssgConverter works with either Python
2.x or 3.x, 3.x is about twice as fast, but 2.x uses about half as
much memory. If you have less than 16 GB of RAM, you will want to stick
with 2.x. If you have more than 16GB of RAM, you should be fine with
3.x. If you have exactly 16GB of RAM, you may want to install both.
My Setup: I have 16GB of RAM and I installed the latest 3.x for Windows
but then I also installed CYGWIN which comes with Python 2.x by default.
For pretty much everything, I use 3.x, but there are 3 large texture
files that I can't extract with 3.x because my machine runs out of
memory. For those 3 files, I can extract them using CYGWIN with
python2
C. Usage:
pssgConverter.py [options] infile [outfile] [pschemaFile]
Flags:
-h : detailed usage/help
-i# : indent XML # spaces (ie: -i4)
-a# : set analysis level to # (0 = default)
-s : safe mode (use if you see warnings)
-f : forcing conversion (even if no change detected)
-o : force create/overwrite pschema (even if it exists)
-p : show progress (if script takes more than 5 sec)
-d : print debug feedback
-t : print trace feedback
-toXML : force conversion to xml (Assume file is NOT XML)
-fromXML : force conversion to pssg (Assume file is XML)
Example: Convert PGZ to XML
$ pssgConverter.py foo.pgz
Extracting [foo.pgz] -> [foo.pgz.pssg]
Converting [foo.pgz.pssg] -> [foo.pgz.xml]
Example: Convert XML to PGZ
$ pssgConverter.py foo.pgz.xml
Backing Up [foo.pgz] -> [foo.pgz.bak]
Converting [foo.pgz.xml] -> [foo.pgz.pssg] using schemaSource [foo.pgz.pschema]
Updating [foo.pgz.pssg] -> [foo.pgz]
You can also specify your own filename and/or schema:
Example: Convert PGZ to XML using Analysis Level 1 and Safe Mode with custom XML and schema:
$ pssgConverter.py -a1 -s foo.pgz custom.xml custom.pschema
[INFO] : Analysis Level [1]
[INFO] : SafeMode Enabled
Extracting [hea_huf_hybride_03.pgz] -> [hea_huf_hybride_03.pgz.pssg]
Converting [hea_huf_hybride_03.pgz.pssg] -> [custom.xml]
Example: Convert XML from previous example to PGZ:
$ pssgConverter.py custom.xml newfile.pgz
Converting [custom.xml] -> [newfile.pgz.pssg] using schemaSource [custom.pschema]
Updating [newfile.pgz.pssg] -> [newfile.pgz]
D. Details:
Reads in PSSG and attempts to make an educated guess at the files schema.
Stores schema to file as
pssg file to XML based on the pschema.
If a
This allows users to make adjustments on how chunks of binary data are
displayed.
The following types are supported by pschema files:
TYPE Supported Aliases Size
----------- ------------------ -----
BYTE byte 1 Byte
BOOLEAN boolean 1 Byte
INT32 int 4 Bytes
UINT32 uint 4 Bytes
FLOAT float 4 Bytes
SHORT short 2 Bytes
USHORT ushort 2 Bytes
STRING string, str, char[] N Bytes
VECTOR vector 12 Bytes
BYTEARRAY byte[] N Bytes
INT32ARRAY int[], int32[] N*4 Bytes
UINT32ARRAY uint[], uint32[] N*4 Bytes
FLOATARRAY float[] N*4 Bytes
SHORTARRAY short[] N*2 Bytes
USHORTARRAY ushort[] N*2 Bytes
Within the pschema file, if you assign a type, you can use a single string
such as:
"_Type" : "INT",
You can also provide a type definition with an array of types:
"_Type" : ["FLOAT","FLOAT",,"FLOAT","INT"],
We could also define the Type above like so:
"_Type" : ["FLOAT[3]","INT"],
Bound Arrays: A bound array is an array with a finite size. With the
exception of STRING, all types support bound arrays. Above,
Float[3] is a bound array. It means 3 Floats.
Unbound Arrays: All types except STRING, VECTOR and BOOLEAN support
unbound arrays. For example:
"byte[]". <- All (remaining) content would be displayed as bytes (hex).
Unbound arrays can be used independantly or appear at the end of a type
definition. If any other types follow an unbound array in a type
definition, those types are ignored. STRING is considered an unbound
array.
You will need the .pschema used to decode the original PSSG to xml in
order to convert the XML back to PSSG. This script will look for it in the
same directory as the XML file. While you can always generate the pschema,
any custom tweaks would be lost so make sure to backup any custom edits.
Analysis Level: Option to attempt to guess at the structure of binary
values that normally render as hex. How much time is spent guessing is
determined by the level. Max (implemented) level is 1.
SafeMode: If you see warnings when converting from XML back to PSSG then
you need to re-extract the offending pssg to xml with this flag. It adds
"_
issues when converting from XML back to PSSG
For Even more detail on PSSG format, view the python file pssgConverter.py
A breakdown of The schema is provided within.
E. Batch Processing:
Batch processing refers to processing an entire directory instead of a
single file. This distro includes some WINDOWS BATCH files that help users
who aren't comfortable with command line tools.
Basically you throw pssgConverter.py into a directory and then also copy
in one or both of the Batch Files:
1) pssgConvertToXmlBatch.cmd
2) pssgConvertFromXMLBatch.cmd
You can then Double-Click the .cmd file in explorer and it will open a
console and kick off, applying the pssgConverter.py script to any
compatible files in the directory. The "toXML" version looks for a
specific list of well known file extensions that are either compressed
PSSG or uncompressed PSSG. The "fromXML" version simply looks for any and
all XML files, backs up the original file and then re-writes the original.
The fromXML version also performs a basic timestamp check and skips any
files that look like they haven't been modified since creation.
C:\path\to\datalocal>pssgConvertToXmlBatch_r.cmd
If you happen to have console open, you can also run the batch files with
flags that will get applied to all the files. So if you want Analysis
Level 1 applied, xml file indention to be 2 and you prefer safeMode for
all XML, you can run it like so:
C:\path\to\datalocal\dir>pssgConvertToXmlBatch.cmd -i2 -a1 -s
Also remember that you can run scripts easily from CYGWIN without needing
the batch scripts. For example:
$ for f in *.xml ; do ./pssgConverter.py $f; done
There are also RECURSIVE versions of the batch files under the directory
batch\recursive. These process the current directory AND subdirectories.
So if you just wanted to extract EVERYTHING (keep in mind, this will more
than double the amount of space things take on your computer), you could
run the recurisive version from the root directory:
And about... 8 hours later everything would be extracted (I'm not kidding)
If you do decide to extract everything, I recommend installing Python 3
into CYGWIN, updating your python symbolic link to use it (See GreedFall
Modding Tips) and run it from cygwin. I find the cygwin python 3 is about
1.5 times faster than the windows version and ~3X faster than python 2.
F. Drag N Drop
For users who do no like console, you can copy pssgConverterDragNDrop.cmd
to the same directory as pssgConverter.py. If you drag any file onto
pssgConverterDragNDrop.cmd, it will open a console and pass the filename
on to pssgConverter.py. It also pauses once the command is complete
so you can read any feedback incase there was an issue.
G. Notes:
- If you are using Python 32 bit, you may run into memory errors on
some files (texture and level files can get big)
- If you have CYGWIN, but you use the batch .cmd file, CYGWIN will
kick off a windows CMD shell to run the file, which will use the
windows version of Python you have installed. So if your windows
version of python is 32 bit and your CYGWIN version is 64 bit,
using the .cmd file may limit the memory available to the script.
H. Compatibility with EGO Editor
The XML is not compatible with EGO, but the PSSG is (assuming EGO can
handle the PSSG). PSSG Converter labels all XML output with the version
2.3.0 header. EGO labels with 1.0.0. PSSG converter thus checks the
version when you work with XML and will reject EGO exported XML files.
-------------------------------------------------------------------------------
II. > > > > PSSG Extract DDS
-------------------------------------------------------------------------------
A. Overview
pssgExtractDDS.py decompresses .pgz/stxu/stxh files to .pssg, then
extracts image data and slaps a .dds header on them so that other apps
can open the file.
.pgz : Low Res Texture (bad extension as it is used by other things)
.stxh : High Res Texture
.stxu : Ultra Res Texture
B. Prerequisites:
You will need to install Python 2.x or 3.x and use 7-zip to extract the
.spk files to a directory (outside greedfall) to expose the games
resources. Within you will find directories named "textures".
Unlike pssgConverter, pssgExtractDDS is optimized to work with
large files without memory issues (since I knew when I made it that
I would be processing large texture files)
C. Usage:
pssgExtractDDS.py
pssgExtractDDS.py
pssgExtractDDS.py
Flags:
-h : detailed help + notes
-d : show debug feedback
-t : show trace feedback
Example: Convert PGZ to .dds:
pssgExtractDDS.py somefile_diff.pgz
Output:
D. Details:
The safest bet is to unpack all .spk files found under GreedFall/packs
to a common directory outside the game folder. (You don't want to unpack
within the game folder, or it may try to load everything as overrides a
nd make the game unplayable).
The most note-worthy package for texture developers is:
datapchighres.spk
As the name implies, it contains most of the "ultra" quality textures.
They can also be found elsewhere, but a large number are stored here in
a single location. I suspect these are the textures that never get loaded
unless the user has selected Ultra-Settings in their graphics setup.
Most users will want to edit the stxu.dds versions and then scale down
and clobber the lower res versions for older hardware.
E. Batch processing:
Batch processing refers to processing an entire directory instead of a
single file. This distro includes some WINDOWS BATCH files that help users
who aren't comfortable with command line tools.
Using Explorer, copy these two files to a textures folder within Greedfall
pssgExtractDDS.py
pssgExtractDDSBatch.cmd
Then double click on pssgExtractDDSBatch.cmd
This will convert all textures to DDS. The extracted filenames will
look like:
somefile.stxu.
CYGWIN:
You can also use the .cmd file under CYGWIN, however it will use the
windows version of your python installation. If you wish to use the
CYGWIN version of PYTHON in your batch command, you will need to
do it the cygwin way:
Copy pssgExtractDDS.py to the textures folder, then:
$ cd /cygdrive/c/path/to/textures/')
$ for f in * ; do pssgExtractDDS.py $f; done')
F. Drag n Drop:
For users who do no like console, you can copy pssgExtractDDSDragNDrop.cmd
to the same directory as pssgExtractDDS.py. If you drag any file onto
pssgExtractDDSDragNDrop.cmd, it will open a console and pass the filename
on to pssgExtractDDS.py. It also pauses once the command is complete
so you can read any feedback incase there was an issue.
G. Notes:
Latest Gimp (2.10.18) does not handle most the DDS files used by GreedFall
however Blender3D does. You can also download NVIDIAs texture tools
exporter here:
https://developer.nvidia.com/nvidia-texture-tools-exporter
Use pssgMergeDDS.py to convert back to PSSG
-------------------------------------------------------------------------------
III. > > > > PSSG Merge DDS
-------------------------------------------------------------------------------
A. Overview:
pssgMergeDDS.py is a convenience script that injects DDS data into an
existing PSSG file and then recompresses the file using the files original
extension.
The script assumes the input file STARTS WITH the name of the original
file. So if the original file was:
somefile.stxu
Then the script would expect the DDS file to begin with those characters.
If there is extra info between the .stxu and the .dds ... it is just
ignored.
B. Prerequisites:
This script is designed to be used in conjuction with pssgExtractDDS.py
C. Usage:
pssgMergeDDS.py originalfile.pgz.dds')
pssgMergeDDS.py original.stxh.dds')
pssgMergeDDS.py original.stxu.dds')
Flags:
-h : detailed help + notes')
-f : force update (even if dds file hasnt changed)
-d : show debug feedback')
-t : show trace feedback')
Example: Convert somefile.stxu.1024x1024.dds back to somefile.stxu:
pssgMergeDDS.py somefile.stxu.1024x1024.dds
D. Details:
See PSSG Extract DDS above. This is meant to be used in conjuction with
that script.
E. Batch processing:
Batch processing refers to processing an entire directory instead of a
single file. This distro includes some WINDOWS BATCH files that help users
who aren't comfortable with command line tools.
Using Explorer, copy these two files to a textures folder within Greedfall
pssgMergeDDS.py
pssgMergeDDSBatch.cmd
Then double click on pssgMergeDDSBatch.cmd
This will merge any changed textures back into their PSSG files. The
original PSSG file will be backed up with a .bak extension if it
does not exist.
CYGWIN:
You can also use the .cmd file under CYGWIN, however it will use the
windows version of your python installation. If you wish to use the
CYGWIN version of PYTHON in your batch command, you will need to
do it the cygwin way:
Copy pssgMergeDDS.py to the textures folder, then:
$ cd /cygdrive/c/path/to/textures/
$ for f in *.dds ; do pssgMergeDDS.py $f; done
F. Drag n Drop:
For users who do no like console, you can copy pssgMergeDDSDragNDrop.cmd
to the same directory as pssgMergeDDS.py. If you drag any file onto
pssgMergeDDSDragNDrop.cmd, it will open a console and pass the filename
on to pssgMergeDDS.py. It also pauses once the command is complete
so you can read any feedback incase there was an issue.
G. Notes:
- This script INJECTS dds data back into the file it was extracted from.
To do that it needs to know the source filename. Therefore, this script
expects the filename to begin with the name of the source file and that
the original file resides in the same directory. Changing compression
types (BC1 to BC7 for example) has not been tested and may not work. To
be safe, save DDS files with their original BCn (UNORM) settings.
- Latest Gimp (2.10.18) does not handle the DDS files used by ')
GreedFall. However Blender3D does. You can also download the')
NVIDIAs texture tools exporter here: ')
https://developer.nvidia.com/nvidia-texture-tools-exporter')
- Use pssgExtractDDS.py to convert PSSG to .dds files')
-------------------------------------------------------------------------------
IV. > > > > RESTORE BAK
-------------------------------------------------------------------------------
In the Batch directory, there is a windows batch script called restoreBak.cmd
This script was designed to scan the current directory for any .bak files
and clobber their sources.
foo.bak => clobbers foo
foo.pgz.bak => clobbers foo.pgz
foo.pgz.pssg.bak => clobbers foo.pgz.pssg
I needed a script like this when developing the python scripts because
I would often make mistakes and break lots of files. However, most
users shouldn't need this (unless they are messing with the python
code). None the less, here it is incase you need it.
There is also a recursive version under batch/recursive/restoreBak_r.cmd
that applies to the current directory AND drills into subdirectories. Use
with caution as you are more likely to accidently revert something you didn't
mean to with that one.
-------------------------------------------------------------------------------
V. > > > > Version History
-------------------------------------------------------------------------------
1.00: Mixed
Initial release.
1.01: Mixed
Updated script to treat Colors as Floats instead of UINTS.
Updated script so it doesn't open files in read/write mode (to avoid permission issues).
Fixed bug where script excluded unused attributes when writing the pschema. This sometimes caused header errors when exporting XML back to PSSG.
Fixed bug where DEBUG messages (-v) would show the wrong values for VECTORS and COLORS.
Updated Audio Section of Modding Tips
Added Misc -> Levels and Maps Section to Modding Tips.
1.02: Mixed
Updated script to support c-style hex in binary XML tags, mostly to make it easier to paste exported binary data from HxD.
Added section to Modding Tips to addresses Audio (music). How to change and add new music content.
2.00: All Script
Complete re-write of the script so that users can define composite types in pschema files.
Updated the code during the re-write to support both Python 2.x and 3.x.
Added guessing logic to deduce likely structure of binary value chunks.
2.01: All Script
Renamed the script from pssgExtracter to pssgConverter to clarify what the script was capable of
Renamed the Mod on Nexus Mods to "PSSG Tools and GreedFall Modding Tips". PSSG is used by many games.
Added python environment headers to script so it could be ran stand-alone.
Updated documentation and examples to reflect the various name/usage changes.
Added this readme to track changes.
2.02: All Script
Added pssgExportTextures.py
2.03: Mixed
Renamed pssgExportTextures.py to pssgExtractDDS.py
Added pssgMergeDDS.py
Updated Textures section of Modding Tips with walk through on editing textures.
2.04: Mixed
pssgConverter: New custom XML parser (wont barf on bad PSSG xml attribute values)
pssgConverter: Now supports compressed and uncompressed PSSG (no 7-zip necessary)
pssgConverter: Encoding Speed enhancements (about 100x faster)
pssgConverter: Better type guessing/analysis (fixed several edge case bugs).
pssgConverter: New safe mode if you see type warnings.
pssgConverter: Use '-p' to show progress on larger files (default with batch commands)
pssgMergeDDS: Fixed write permission issues
pssgMergeDDS: Encoding Speed is faster
pssgOggExtract: Updated to be Python 3.x compatible
moddingtips: Updated instructions to reflect new features.
moddingtips: Added tip for searching the games .txt files (TymmeRMortis)
moddingtips: Added some content to the items section
added: restorebak script
added: batch processing scripts
added: dragNdrop scripts
-------------------------------------------------------------------------------
VI. > > > > Contributing Authors
-------------------------------------------------------------------------------
Dheu
TymmeRMortis