Sequence Music Engine in KCD

From Nexus Mods Wiki
Jump to: navigation, search

Introduction

To learn about modding of the SFX, please refer to Sound modding in KCD

Portions of this text are taken from a journal article (Sporka and Valta 2017) co-written by the original poster of this article.

For any inquiries, please tweet at @adam_sporka.

DISCLAIMER: Any information on this page is supposed to be used at your own risk :-)

Sequence Music Engine is an adaptive music engine. It is a system for playback of pre-existing music material. It is a C++ library with available bindings to CryEngine (via FMOD Studio) and Unity. SQC runs within a host computer game process. The game communicates with SQC via control variables. (Simple set of key–value string pairs.) The current implementation uses the sampling frequency of 48 kHz and a 2-channel output. SQC is intended to work in a sample-pulling pipeline where an audio engine is requesting buffers of audio to play ("getbuffer operation").

How to mod the KC:D music

TL;DR: Doable but not straightforward.

You need to (1) prepare a .sqcb file which will contain the actual audio data, and (2) modify main_layer.widsh to refer to the newly created .sqcb file.

It's relatively simple for linear songs, such as menu music. It is way more complex for non-linear pieces, such as combat music, battle music, or even simple exploration pieces with seamless transitions. I will elaborate on this topic later on.

I think that the best way to start is to take a look at the 1980s Synthwave KCD Music in the Menu mod (https://www.nexusmods.com/kingdomcomedeliverance/mods/199) and do a diff between the original main_layer.widsh and the modded one. With a bit of hacking and slashing using [1], it is possible assemble the .sqcb file by hand from a raw .ogg file.

Technical Structure of the Soundtrack

A SQC soundtrack is a set of scenes. A scene corresponds to a music piece which is able to function as a standalone music composition. Each scene contains one or more patterns. A pattern is a buffer of audio signal, containing a portion of music. Having multiple patterns in a single scene enables a non-linear playback of the scene’s music material by lining up the patterns in a suitable sequence.

There are three types of patterns, differing in use:

  • Main pattern -- representation of a linear part of the music. The main patterns are the pieces of music material which play most of the time.
  • Branch -- representation of a premature, yet scored ending of the main pattern, typically used when the scene should no longer play, and its expedited termination is required.
  • Intro -- representation of a scored beginning of a scene. Branches and Intros are used in the mechanism called seamless transition.
  • Cinel -- music material which glues two scenes during a transition via cinels.

Every pattern has a known nominal length, corresponding to the length of the musical feature contained in it. For example, a 4-bar pattern at 120 BPM in a 4/4 rhythm has a nominal length of 8 seconds. Its technical duration might be slightly longer in order to enable the playback of overhanging tones or noises. This implements the horizontal re-sequencing technique.

Patterns may have multiple flavors, different versions of the same music material between which the engine cross-fades, based on their individual suitability. This enables the paradigm of adaptive music commonly known as the vertical reorchestration.

SQCB Files

Each scene is stored in a separate SQCB file. Depending on its use, its contents could be one or many SQC patterns. A menu music typically contains just one pattern. A non-linear in-game track (the symphonic parts of the exploration or combat soundtrack) typically contains multiple patterns.

This is the structure of the file:

 Offset      Contents            Comments
 0           L"SQCB"             Fixed header prefix; 16-bit characters
 8           L"1.00"             Version number; 16-bit characters
 16          <32-bit LE int>     Number of files in the bank (N)
 20          Bank items desc
             -- File name        A null-terminated 16-bit string
             -- <32-bit LE int>  Offset of the file in the bank
             -- <32-bit LE int>  Length of the file in the bank
 ?                               Binary data of the files, one after another
 ?           L"RDBP"             End of file suffix

MINI FAQ 1

  • Q: Is there a public packer/unpacker available?
  • A: Not really at this point, sorry, working on it :-)
  • Q: Why are you using .ogg files in Sequence Music Engine
  • A: Ogg is a free and open-sourced format
  • Q: Why "RDBP"?
  • A: Because Rainbow Dash is Best Pony

main_layer.widsh File

The behavior of the music is described in main_layer.widsh, which is the music script file. Here is a commented excerpt. The file shipped with the game is 3,700 lines long. Apologies for its cryptic look. Despite it being a text file, it's generated / compiled from multiple sources of data and was never meant to be looked at, let alone edited, by a human being again.

 SQC MUSIC ENGINE WIDSH FILE VERSION 1.18
 : Any text can be here. I use it for credit information.
 : It won’t affect your music in any way.

 SETUP
 // Line comments have to start with '//'
 // This is run on start-up. Used for variable initialization.

 // A simple assignment looks like this:
 [%string_variable $string_value !]
 // In a C-like language this line would read as
 // string_variable = "string_value";

 [%float_variable #3.14 !]
 // In a C-like language this line would read as
 // float_variable = 3.14;

 // This calculates the circumference of a circle with 10 units radius.
 // Polish Reverse Notation. (Because CD Projekt rulez!)
 [%C [[#2 %float_variable *] #10 *] !]
 END

 UPDATE
 // This is called before each getbuffer operation. It depends on the host
 // application how often this gets called. FMOD Studio on PC typically
 // calls this 5 times during 2 seconds.
 // A music logic is expressed here.
 END

 SCENE GUI/MM-BETA-THEME
 // This is an example of a SCENE block. It provides the metadata for
 // a single scene. The order of the lines is somewhat loose but not that
 // loose, so I recommend to keep it the way it is.
 N MAIN_MENU
 // N: The section in the SQC Player GUI (which is not publically available, sorry)
 L [$MM-BETA-THEME]
 // L: The label shown in the SQC Player GUI
 E [[%menu $on =] [[%closing_credits $on =] [%closing_credits_type $game ~] &] |]
 // E: The selection suitability expression. TL;DR: The expression must evaluate as true
 // if this scene is expected to play and is desired to be chosen.
 P [#16]
 // P: Selection priority
 T [#48000 #600 *]
 // T: Timeout in samples. The example above corresponds to 600 seconds = 10 minutes.
 // The scene is guaranteed not to play for 10 minutes after the completion of its
 // playback.
 S 1 FADE_UNSUITABLE KEEPQUIET @1 1
 // S: Scene mode.
 //   FADE_UNSUITABLE -- As soon as E is false, the playback fades out
 //   KEEPQUIET @1 -- The silence after completion of the playback will last 1 second
 M 1 0 @132 @133 BRANCHES 0
     SYNCPOINTS 65
         96000 +96000 +96000 +96000 +96000    +96000 +96000 +96000 +96000 +96000
        +96000 +96000 +96000 +96000 +96000    +96000 +96000 +96000 +96000 +96000
        +96000 +96000 +96000 +96000 +96000    +96000 +96000 +96000 +96000 +96000
        +96000 +96000 +96000 +96000 +96000    +96000 +96000 +96000 +96000 +96000
        +96000 +96000 +96000 +96000 +96000    +96000 +96000 +96000 +96000 +96000
        +96000 +96000 +96000 +96000 +96000    +96000 +96000 +96000 +96000 +96000
        +96000 +96000 +96000 +96000 +96000
     NOTIFICATIONS 0
 // M <number> 0 <nominal_length_in_samples_or_@seconds> <tech_length>
 //   BRANCHES <number_of_branches>
 //   SYNCPOINTS <number_of_syncpoints>
 //   NOTIFICATIONS <number_of_notifications,_usually_0>
 END

 SCENE EXPLORATION/AFLD4
 // This is an example of an in-game non-linear scene.
 N EXPL_FBK
 E [[%#countryside $on =] [%#quiet $off =] [%#dialog $off =] ^]
 e [[%#countryside $on =] [%#quiet $off =] ^]
 // e: The playback suitability expression. As long as this is true, this scene can
 // play. If not present, the default is true.
 L [$OPEN_LANDSCAPE_(AFLD4)]
 S 1 SEAMLESS 1
 // S: Scene mode.
 //   SEAMLESS -- the seamless transitions mode
 E [[%#countryside $on =] [%#quiet $off =] ^]
 // E: Suitability expression applied for the next patterns.
 M 1 0 4799052 4943052
     BRANCHES 9
         21 432000 22 864000 23 1296000 24 1872000 25 2304000 26 2880000 27 3456000
         28 3935053 29 4367053
     SYNCPOINTS 122
         36000 72000 108000 144000 180000 216000 252000 288000 324000 360000
         396000 468000 504000 540000 576000 612000 648000 684000 720000 756000
         792000 828000 900000 936000 972000 1008000 1044000 1080000 1116000 1152000
         1188000 1224000 1260000 1332000 1368000 1404000 1440000 1476000 1512000 1548000
         1584000 1620000 1656000 1692000 1728000 1764000 1800000 1836000 1908000 1944000
         1980000 2016000 2052000 2088000 2124000 2160000 2196000 2232000 2268000 2340000
         2376000 2412000 2448000 2484000 2520000 2556000 2592000 2628000 2664000 2700000
         2736000 2772000 2808000 2844000 2916000 2952000 2988000 3024000 3060000 3096000
         3132000 3168000 3204000 3240000 3276000 3312000 3348000 3384000 3420000 3492000
         3528000 3564000 3600000 3636000 3672000 3708000 3744000 3781732 3824620 3875640
         3971052 4007052 4043052 4079052 4115052 4151052 4187052 4223052 4259052 4295052
         4331052 4403052 4439052 4475052 4511052 4547052 4583052 4619052 4655052 4691052
         4727052 4763052
     NOTIFICATIONS 0
 // A main pattern 4799052 samples long with 9 branches to endings nr. 21, 22, 23, etc.,
 // with 122 synchpoints placed at 36000, 72000, ... samples starting from 0.
 // Its number is 1. The corresponding .ogg file in the AFLD4 is called AFLD4-01.ogg
 E [#1.0]
 I 10 0 288000 432000 0 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 I 11 0 288000 432000 1 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 I 12 0 288000 432000 2 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 I 13 0 288000 432000 3 SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000
 // I: An intro of a given color. 0 = white, 1 = red, 2 = green, 3 = blue.
 // For details please read the paper [1] linked below in References.
 // Intro 10 would be AFLD4-10.ogg
 B 21 0 288000 432000 
     BPOINT 144000 TO_ALEPH 2 
     SYNCPOINTS 7 36000 72000 108000 144000 180000 216000 252000 
     NOTIFICATIONS 0
 // B: A branch, contained in AFLD4-21.ogg
 B 22 0 432000 576000
     BPOINT 144000 TO_ALEPH 2
     SYNCPOINTS 11 36000 72000 108000 144000 180000 216000 252000 288000 324000
         360000 396000
     NOTIFICATIONS 0
 ...
 END

MINI FAQ 2

  • Q: Does the file need to be called "main_layer.widsh"?
  • A: Unfortunately, yes, for now. At some point I'll add a CVar allowing you to specify a different filename.
  • Q: What happens when I change the file while the game is running?
  • A: You need to restart the game in order to hear the changes.
  • Q: What happens when I replace the .sqcb file while the game is running?
  • A: Good question, TBH, I didn't need to try that. In theory, next time the scene is loaded for playback, its new contents should be played back. However, if you need to provide a different timing information, you need to specify that in main_layer.widsh and that one won't be reloaded until the end of the KingdomCome.exe process.
  • Q: Is there a better way how to debug the music than restarting the game every single time?
  • A: We have an in-house playback tool. It hasn't been released to the public yet.
  • Q: Why is it so god-damn complicated?
  • A: These are the internals of the engine. In an ideal world, I'd provide a neat GUI and a WYSIWYG (WYSIWYH?) editor. I did develop a GUI interface for the soundtrack development I'd rather sit on it until it's ready for a public release. Thanks for your understanding!
  • Q: Did you really spend all that time typing all the numbers?
  • A: Hell no! I wrote a tool which generated those too. Looks intense but once we got used to our workflow, adding more music was relatively easy.
  • Q: What does WIDSH stands for?
  • A: Well-Informed Data Structure Header. Actually, that's not it. But it's not at all important. I was serious about Rainbow Dash though!

Reference

  • Sporka A, Valta J (2017) Design and implementation of a non-linear symphonic soundtrack of a video game. New Review of Hypermedia and Multimedia. Volume 23, 2017, Issue 4. Taylor&Francis. https://www.tandfonline.com/doi/abs/10.1080/13614568.2017.1416682 Please tweet at @adam_sporka if you wish to obtain a copy of the paper.