Jason Booth
Wow! That was a lot of theory to take in. Now we're ready to dig into using the program that does it all: DirectMusic Producer. DirectMusic Producer is a complex program with a steep learning curve. While this section is not meant to be artistic, it demonstrates DirectMusic's basics, as well as the fundamentals required to use the program in a production environment. We also revisit some basic ideas key to using DirectMusic, including variation, styles, and chord changes. Our goal is to create a simple blues that uses DirectMusic's recomposition features to follow chord changes.
Getting Started
Insert the CD included with this book into your CD-ROM drive, as we refer to MIDI files on this CD for the note data in this chapter. Open DirectMusic Producer and create a new project by selecting File>New from the menu and selecting a project in the dialog box. Name your project, and choose a storage location for its data on your hard drive. Select File>Import File into Project>MIDI File as Segment from the menu. Navigate to the Unit I\Chapter 5\Tutorial Part One directory on the CD and select the file Tutorial_Bass_Melody.mid. The Segment opens automatically in the main screen of DirectMusic Producer.
A DirectMusic Segment file, created from a MIDI sequence.
This Segment has several tracks of data in it. The first is a Tempo Track, which controls the speed of playback. The second track is a Chord Track, which controls how DirectMusic transposes other Segments over this Segment. The next two tracks are Sequence Tracks containing note data. The final track is a Band Track, which controls the instrument patching.
Often, when importing data from sequencing programs, the data requires some amount of cleanup. In this case, there are several problems; first, the Segment is two bars long, yet the music in the Sequence Tracks is only one bar long. To fix this problem, right-click on the title bar of the window and select Properties.
The properties dialog box.
Note Almost everything in DirectMusic has a properties dialog; our Segment itself has one, as well as each track and each piece of data in each track.
Click on the button next to Length, and set the number of measures to one.
Patterns
The music we imported from the MIDI file is incredibly simplistic. It is a single bar of a bass line and melody and hardly interesting at that. However, music often exists as the collective of small patterns like this one, which play over various chord changes. Our next step is to make this small piece of music play over a series of chord changes, using DirectMusic's transposition functionality to adjust the notes to match the new chords.
Currently, our music is stored as a Sequence Track in our Segment. Sequence Tracks do not transpose to follow chords, and play exactly as the author wrote them. To get our melody and bass line to transpose over chord changes, we need to use a Pattern Track instead.
To create a new Pattern Track:
Select File>New.
Set the number of measures to one.
Select Pattern Track from the menu.
Select the Segment1.sgp (which was created) from the menu on the left-hand side.
Right-click and select Rename.
Rename the Segment to BassSegment.sgp.
We need to copy the data from our old Segment into the new Segment's Pattern Track. Select the Tutorial_Bass_Melody.sgt Segment and maximize track one's window. This opens the roll editor, where you can view and edit note data.
The roll editor displays note data, which you can edit right in DirectMusic Producer.
Select the first note in the window and press Ctrl+A to select all the notes. Press Ctrl+C to copy this data. Open Pattern Track one in our new Bass pattern. Click on the first beat in the measure, and press Ctrl+V to paste the data into the track. You may have to scroll the track up or down to see the note data.
Create a new Segment named Melody.sgt with a Pattern Track, and repeat this process with track two. At the end of the process, you need to point the Pattern Track to channel two by right-clicking on the Pattern Track to bring up the properties dialog box and set the pchannel to two.
Setting the Chord Reference
For DirectMusic to transpose a melody to a new chord, it needs to understand the relationship of the melody to its chord and scale. In our case, we base the bass line and melody off a C7 chord.
Add a Chord Track to the bass and melody Segments by right-clicking in the Segment window and selecting Add Track and Chord Track from the menus. Right-click on the first beat in the measure of the Chord Track and select Insert to insert a chord.
Figure 5-4: We define the chord and scale reference in the Chord Properties page. Here we use a C7 chord with a C Mixolydian scale.
On the bottom piano roll, set the chord to C7. Do this by clicking on the appropriate notes on the piano roll (C2, E2, G2, and Bb2). Now adjust the scale by changing the B natural to a Bb. Repeat this process for the second part as well.
Note You can copy the chord from the Chord Track and paste it in the other Segment's Chord Track, rather than resetting the notes manually.
Creating a Primary Segment
Now that both of our parts reference chords and scales, we must create a chord progression for them to play over. Create a new Segment called MasterSegment.sgt, set its length to four bars, and add a Chord Track. Place a series of chords in this Chord Track for the bass and melody parts to follow by selecting bar one and pressing Insert to insert a new chord. Use the following chord progression:
C7> F7> G7> C7
The notes and scales for these chords are shown here:
Chord
Chord Notes
Scale
--------------------------------------------------------------------------------
C7
C E G Bb
C D E F G A Bb
F7
F A C Eb
C D Eb F G A Bb
G7
G B E G
C D E F G A B
Adding your melody and bass line.
Add two Segment Trigger Tracks to MasterSegment.sgt. A Segment Trigger Track allows you to include another Segment inside of your Segment, which is extremely useful for organization. In your Segment Trigger Track, right-click on the first beat of the first bar and select Insert from the menu. Select your bass Segment from the drop-down box in the properties page. Now copy and paste this Segment into all four measures on the Segment. Repeat this process on the second track to include the melody as well. Play this Segment. Notice that your instrument settings vanish, and both tracks sound like a piano. To fix this, add a Band Track to the Segment, and copy the Band from your original, imported Segment file into this track.
When you are done, MasterSegment.sgt should look like this:
Your completed master Segment.
Play the Segment and notice how the notes now transpose to follow the chord progression.
Making Our Music Cooler
Play the Segment and notice that both the bass and melody move in a linear fashion. That is, when the chord changes from C7 to F7, both lines move up a fourth in parallel. If two musicians were to play this, it is unlikely that they would move through these changes in such a strict parallel fashion. DirectMusic allows us a number of ways to specify how a given part should move through the chord changes.
Open the properties page onto the F7 chord. Notice that there are four different chord and scale levels. Use these different levels to specify alternate ways in which DirectMusic should interpret the chord. On the second level of the chord, change the chord mapping so that the top two notes of the chord are on the bottom of the chord. It should now be composed of C, Eb, F, and A, which is the second inversion of the chord. For the G7 chord, set the second chord to the second inversion as well (in this case, C, E, G, and B). Your chord should look like this:
Setting up alternate ways for DirectMusic to interpret the G7 chord.
Now go back to your melody Segment file, right-click on an area of the pattern where no notes are found, and bring up the properties dialog. In the bottom, right-hand corner under Default Play Mode, set the chord level to Chord Level 2 so that this pattern follows the chord mappings specified in the second layer of the chord.
Adjusting chord levels.
Play MasterSegment.sgt again. Notice that while the bass is moving linearly through the chord change, the melody now uses the inversion to determine the root note of the melody. Save your project.
In case you are lost, there is a completed version of this section of the tutorial on the companion CD located in Unit I\Chapter 5\Tutorial Part One.
Variations
Now that we have a basic piece of music playing over some chord changes, we want to add some variation to the playback so that each time our bass line plays, it plays a little bit differently. Open the BassSegment.sgt that we created earlier, and open the Pattern Track. Notice the variation bar. By default, DirectMusic Producer fills in all of these variations with the same data.
Figure 5-9: The variation bar.
Select only the variation to work on. Notice in the above figure that we selected all variations because every number in the bar is depressed. You can quickly select or unselect all variations by clicking the vertical bar on the leftmost edge of the variation bar. Click this bar so that every variation number is not depressed, then select variation 2. Your variation bar should look like this:
Only variation 2 is selected.
In the pattern, raise the note on the second beat to an E, and lower the last note to an E. Your pattern should look like this:
Our first bass line variation.
If we play the project now, we would only hear this variation one out of 32 times, since DirectMusic fills out our 32 Segments for us by default. To have the variation play more often, disable all variations except for variations 1 and 2 by deselecting variation 2 and selecting variations 3 and higher. Right-click on the Variation tab, and select Disable. Your Variation tab should now look like this:
Variations 1 and 2 are enabled, while all others are disabled.
Play the master Segment, and notice how both our original bass line and our variation play.
Again, we saved a completed version of this section of the tutorial on the companion CD in the folder Unit I\Chapter 5\Tutorial Part Two.
Styles
In the previous two sections of this chapter, we stored our musical data in Pattern Tracks. Pattern Tracks have a lot of power, as they can follow chord changes in multiple ways and contain up to 32 variations. However, even more powerful than patterns are Styles. As defined in Chapter 4, a Style is a collection of patterns, which DirectMusic chooses based on various parameters. Each pattern within a Style acts just like Pattern Tracks, containing chord mappings and variations. Moreover, because you can place multiple patterns within a Style, they allow you an unlimited number of variations. More importantly, they respond to the groove level.
While groove level can be used in many ways, it is easiest to think of it as an intensity filter. Each pattern within a Style has a groove range in which it is valid. Our melodic line might be simpler at groove levels one through ten, while a more active one is used from 11 to 20. When the Style is choosing the pattern to play, it looks at the current groove level and determines which patterns are valid within the Style. A typical game implementation involves tying groove level to the intensity of the current game situation. If there is a lot of action going on, the groove level raises and the music intensifies, and when the action dies down, the groove level lowers and the music calms down.
Creating Our First Style
Create a new Style by clicking File>New and selecting Style from the menu. In the left-hand menu, open the folder in the Style named Patterns. Double-click on the pattern to open it. Using the same techniques we used in the first tutorial in this chapter, copy the data from our melody pattern to this Pattern Track. You will need to set the pchannel for this part to two, as well. Now select the pattern in the left-hand menu, and press Ctrl+C to copy it into memory and Ctrl+V to paste it. Your Style should now have two patterns called pattern one and pattern two. Open pattern two, and adjust the notes to make the melody into more of an accompaniment Style line. You can use the right-click menu on the mouse to add new notes if you need to. Our accompaniment looks like this:
Our accompaniment pattern.
Rename the Style Piano.
Open our MasterSegment file, and use the right-click menu to delete our current melody Segment Trigger Track. Use the right-click menu to add a Style Track. Right-click on the first bar in the Style Track, and select Insert. Select our Piano Style from the menu.
Press Play on the master Segment and notice that both patterns are chosen from our Style. Also of note is that we do not have to place references to our Style in every measure of the master Segment. This is because a Style Track is assumed to repeat, unlike our Segments, which have a fixed length.
Adding Groove Level
Go back to our Style, and bring up the properties on pattern two. Notice that there is a Low and High setting under Groove Range. This sets which groove levels this pattern is valid. Set these numbers to 1 and 10, respectively. This will make this pattern only viable from groove levels one to ten.
Now select pattern one, and set its groove range from five to 100.
Go back to our master Segment, and use the right-click menu to add a new track. This time, select Groove Track from the menu. Select the first bar in the Groove Track and press Insert.
Play the master Segment, and notice that only pattern two is used. This is because our groove level is set to 1, which is not a valid groove level for pattern one to play. Now set the groove level to 6 in our Groove Track, and play the master Segment. You will notice that both patterns now play. This is because both patterns are valid, so DirectMusic will choose between them every measure.
Set the groove level to 11 in our Groove Track. You will notice that only pattern one plays, because pattern two's groove range is too low to play. Save your project.
The wonderful thing about using groove levels to control your music is that it's very easy to work with and test on the musical end and provides a simple mapping for a game to use that doesn't require your programmers to understand how your music is composed. Often, groove level can be tied to something very simple for a game engine to understand, such as the number of monsters in the area, and the composer can decide what the music should do in these cases.
Once again, we saved a completed version of this section of the tutorial on the companion CD in the folder Unit I\Chapter 5\Tutorial Part Three.
This chapter offered some insight into the basic mechanics of using DirectMusic Producer. While DirectMusic Producer can seem daunting at first, once you become familiar with the interface, you will find it efficient and well designed. Remember to start small and be prepared to build several projects as a learning experience.
Thursday, December 20, 2007
Variation
Scott Selfon
Okay, we understand linear pieces of music and getting from one piece of music to another. A primary challenge to overcome is repetition. No matter how smooth our transitions, if we hear the same pieces of audio repeatedly, they are bound to get stale. Here is where DirectMusic provides significant strength!
Variable Playback/Variations
Content variability can be achieved on several different levels. On a "microscopic" level, content can specify ranges for volume and pitch on individual waves played in Wave Tracks, as well as randomized volume, start time, and duration on notes in Pattern Tracks.
The Wave Properties page allows you to specify ranges for attenuation and fine-tuning. This particular example will randomly pitch-shift the wave up or down by two semitones.
The Pattern Note Properties page allows you to specify variability for velocity, start time, and duration. This particular note will play with randomization in start time, duration, and with a random velocity between 75 and 125.
On the "macroscopic" level, you could of course author multiple Segments and write a script to choose between them randomly.
Then we have variations, which provide the best of both worlds. Variations are those 32 intimidating-looking buttons present on Wave Track parts and Pattern Track parts (in Segment files), as well as in pattern parts (in Style files, a format which we cover in a bit).
Each of those buttons represents one version of the part; for instance, you could have a drum part where variation 1 includes the full drum part, variation 2 removes the snare, variation 3 mixes up the rhythms a bit, and so on.
The primary thing to remember about variations is that at any given time, a single variation plays per part. This does not mean that every single part is playing variation 12 at the same time, but it does mean that if you author 12 variations for your drum Track, only one is being played at any given time. Each part chooses a new variation to play whenever its Segment is started or loops.
Before we get into the details of variation behavior and functionality, let's talk a bit about the 32 buttons, which can be among the trickier aspects of DirectMusic Producer to understand. When you first create an empty part that supports variations, each of the 32 button labels is white. This indicates that these variations are empty. As soon as you add any note, continuous controller, or wave information to a variation, that variation's label becomes black, which indicates that it has some information in it.
One potentially confusing aspect of variations is that empty variations are just as acceptable to be "picked" to be played as variations with data. For instance, a composer might create three alternate versions of a drum part on variations 1 through 3. When he then tries to play the piece of music, he finds that most of the time it's silent. Why? Because variations 4 through 32 were enabled and empty (silent), meaning 29 out of 32 times, an empty variation was chosen.
This brings us to disabling variations. When a variation is disabled, DirectMusic will never "pick" it to be played. By default, all variations are enabled. This is indicated in DirectMusic Producer by a solid gray background for the button.
which typically should be deleted (since the variation will never be chosen).
To disable one or more variations, make sure the appropriate variation is pressed by clicking on it; then right-click anywhere over the 32 variation buttons and choose Disable from the right-click menu that appears. Disabled variations have a checkerboard look to them. Note that it is possible to have a non-empty variation that is disabled. In most typical scenarios, this means that you are saving data that you will never use, so it is generally a good idea to go ahead and erase any disabled variations when your project is completed.
Auditioning Variations in DirectMusic Producer
The final aspect of the user interface for variations is the buttons being selected ("pressed down") or not selected ("up").
The main thing to remember about this aspect of the buttons is that it is solely for editing and auditioning purposes in DirectMusic Producer — whether a button is pressed or not has no bearing on how the variations will behave in a DirectMusic player or in a game at run time. Pressing buttons provides for two purposes:
Editing one or more variations simultaneously. Pressing a single variation button means that you are now editing only that variation. Paste from the clipboard, add a note, or drag a wave into the part, and that data is assigned solely to that variation. If you select multiple variations, notes and waves added will be identically added to each of the selected variations.
Auditioning variations within DirectMusic Producer. Let's say you created a four-part piece of music, and each part had ten enabled variations. You listen to it a few times and notice that there's an incorrect note in one variation of the bass part. Rather than forcing you to keep repeatedly playing the Segment until that variation gets picked again, DirectMusic Producer has an audition mode, where the same variation (whichever one is "pressed") will be chosen every time the Segment is played. Only one part can be in audition mode at a time. To select a part for audition mode, just click anywhere on it. The part that is in audition mode is highlighted in a slight yellow shade.
Part three (piano) is in audition mode. Within DirectMusic Producer, all other parts will randomly choose which variation to play from the enabled variation buttons. Part three will always play whatever variation button is pressed (until it is no longer in audition mode).
Remember that audition mode only applies in DirectMusic Producer; when a Segment is saved and played in a DirectMusic application, none of its parts will be in audition mode, and all parts will choose from their enabled variations.
Audition mode provides for some unique behaviors not seen outside of DirectMusic Producer. First, as multiple variation buttons can be selected ("pressed") at once, you have the opportunity to hear a single part playing multiple variations at once. As mentioned previously, only a single variation will be chosen at a time in an actual DirectMusic application. You can also select disabled variations and audition them; a DirectMusic application will never play a disabled variation. Lastly, you can have all of the variations on your auditioned part deselected, in which case no variation will be chosen, causing this part to be silent. Again, this behavior is solely an audition feature of DirectMusic Producer.
Variation Shortcuts
There are a few shortcuts provided for quick selection of which particular variations you are editing. The vertical bar to the left of the 32 variation buttons selects (and with a second click deselects) all enabled variations. Double-clicking on a variation will "solo" it (only select that variation). Double-clicking again will "unsolo" it, reverting to the previously pressed buttons.
If you are using linear wave files, a shortcut is provided to avoid needing to work directly with the 32 variation buttons at all. From the File menu, choose Import>Wave File(s) as Segment Variation(s)…. From the dialog that appears, choose all of the wave files that you want to be variations in your Segment. A clock-time Segment is automatically generated, with a length of the longest wave file you chose and using one variation per wave file. You'll notice that in this case, all of the empty variations (unless of course no variations are empty) are automatically disabled.
Specifying Variation Behavior Settings
Once you have created a Segment that utilizes variations, the next logical step is to be able to more finely control how variations are chosen. For instance, if we had 20 variations of a dialog fragment, we would really like the person hearing this Segment not to hear the same variation played twice in a row. Actually, it would often be ideal to play every possible variation once before we ever get a repeat. This kind of functionality can be specified by the audio producer from each part's properties page.
As usual, right-click a part and choose Properties (or if the Properties page was already open, click the part to make it the active property display).
Figure 3-8: Selecting per-part variation behavior for a Pattern Track part.
Figure 3-9: Selecting per-part variation behavior for a Wave Track part.
Though presented in different locations on the Pattern Track and Wave Track property pages, the available options and functionality are the same:
In random order: The default behavior — a new variation is randomly selected from the enabled variations every time the piece of music plays or loops. Repeats can occur.
In order from first: The first time the Segment is played, variation 1 (or the first enabled variation) is played. On each repeat or loop, the next enabled variation in order is played.
In order from random: The first time the Segment is played, a random enabled variation is chosen. On each repeat or loop, the next enabled variation in order is played.
With no repeats: Random enabled variations are chosen on each repeat or loop, but the same variation will not be chosen twice in a row.
By shuffling: On each repeat or loop, a random enabled variation is chosen, but every enabled variation will be played once before any are repeated.
Recall that a new variation is chosen each time a Segment is played or it loops. However, there are situations where we might want to "reset" the variations. For instance, let's say that each variation is the next step up in an ascending scale, and we allow the Segment to loop as the character picks up tokens. If they haven't picked up any tokens in the last few minutes, we want to stop the Segment and start the scale over from the lowest note the next time a new token is picked up. This is where the Reset Variation Order on Play option (available in The Wave Track and Pattern Track property pages) can be used to control whether DirectMusic "remembers" which variations were previously played the next time the Segment starts. If, for instance, a Segment was set at In Order from First and its Pattern Track had Reset Variation Order on Play checked, every time the Segment was played, it would start from variation 1. If it looped, it would then play variation 2, 3, and so on.
Reset Variation Order on Play functionality for the Pattern Track.
There are, of course, instances where the audio producer would prefer for two or more parts to always pick their variations identically. For instance, in a DirectMusic arrangement of a jazz trio, there might be several "solos" for each of the instruments (parts). We do not want two instruments to "compete" by playing their solos at the same time. For these kinds of situations, Lock ID functionality provides a way to control how different parts choose their variations. Every part that has the same Lock ID will choose its variation to be the same. Therefore, for the above scenario, if each part has three solos to choose from, we could place the drum part's solos on variations 1-3, the guitar's solos on variations 4-6, and the piano's solos on variations 7-9, for example. For the variations where the parts do not have solos, they would either "lay out" or just keep time. When the Segment was played, each of the parts would choose its variations in step with the other parts, so only one solo would be played at a time.
This Wave Track part's Lock ID is set to 1. Any other Wave Track parts in the Segment that are set to Lock ID 1 will choose the same variation as this part.
Note that Lock ID does respond to the other settings that we've discussed that control how variations are chosen; if one locked part has variation 10 disabled, that variation will never be chosen for that entire Lock ID group. Similarly, if one part is set to play its variations in order, all of the parts will follow that part's lead. Mixing and matching between various part variation settings (for instance, one part set to shuffle, another part set to play in order) within a Lock ID group can lead to somewhat unpredictable variation choices, so it's generally recommended that all parts in a Lock ID group use the same settings.
Another important concept to remember is that Lock ID is unique per Track, not per Segment. That is, several parts in a Pattern Track that use the same Lock ID will choose the same variation. However, a Pattern Track part and a Wave Track part using the same Lock ID will not lock to each other. A limited subset of this kind of functionality is available in the Wave Track's property page with the Lock Variations to Pattern Track option. Checking this box means that each pchannel of the Wave Track is now locked to the corresponding pchannel of the Pattern Track; a Wave Track part on pchannel 1 will always choose the same variation as the Pattern Track part on pchannel 1 if one is present.
Locking a Wave Track's variation to a Pattern Track.
Locking a Wave Track's variations to a Pattern Track is most useful when you wish to use MIDI continuous controllers to manipulate waves — panning, MIDI volume control (fade in, fade out, etc.), and so on. By authoring corresponding variations in the Pattern Track, you can ensure that a particular wave plays using particular MIDI controller information. Alternatively, you could keep the Wave Track unlocked from the Pattern Track, in which case any of your controller variations might be used over any Wave Track variation.
Variation Switch Points
As we've already discussed, a new variation is chosen for each part whenever that part's Segment starts playing or loops. But what if we want variations to be chosen more frequently? In particular, the audio producer might create small fragments of melodic or rhythmic lines that he wants to be able to switch between in the midst of a Segment. Variation switch points provide the audio producer with a solution to this — namely, the ability to determine if and when a part can change variations in the middle of playback.
Variation switch points are displayed in DirectMusic Producer in a separate strip for each Pattern Track part (they are not supported by Wave Track parts). The two types of switch points are enter switch points and exit switch points. Enter switch points tell DirectMusic where it is legal to "jump into" this particular variation. Exit switch points tell DirectMusic where it is legal to try to find another variation to change over to. If no variation has an enter switch point corresponding to this exit switch point, the currently playing variation will continue to play.
A variation using variation switch points. This variation can jump to any other variation at the position of the exit switch point (red blocks), shown as the dark blocks in the Var Switch Points line. Similarly, this variation can start playing mid-Segment if any other variation has an exit switch point corresponding to this variation's enter switch point (green blocks), the lighter blocks in the Var Switch Points line.
While opening many possibilities for potential variation mixing and matching combinations, variation switch points can make composition much more challenging. Rather than only having to worry about how different variations meld at Segment loop points, variation switch points open up the possibility of playback jumping between variations right in the middle of playback, at whatever boundaries they wish. Often it is easiest to limit variation switch points to measure boundaries, although they do support down to grid resolution if you need it.
Influencing Variation Choices
DirectMusic provides even more acute functionality for controlling which variations play when, particularly when the audio producer is using chord progressions. We cover the various aspects of chord proressions (chords, Chord Tracks, and ChordMaps) in more detail in Chapter 6, but for now just know that a composer can define a static chord progression by placing chords in a Chord Track or a dynamically generated progression by using a ChordMap and a Signpost Track. Regardless of how the chords generate, a composer can elect to have variation choice influenced by the function of the currently requested chord in the current key. This functionality is in the Variation Choices window, more informally known as "The Mother of All Windows." The window opens from the button to the right of the 32 variation buttons:
Figure 3-14: The Variation Choices window.
In this window, each row corresponds to one of the 32 variations. Each column allows the audio producer to specify exactly what kinds of chords a variation will or will not be chosen for. For instance, you might have a melodic line in a variation that sounds great played over a IV chord (e.g., an F major chord in the key of C), but you have another variation that provides a stronger option to play on a ii chord (e.g., a D minor chord in the key of C). By selecting and unselecting the various buttons in the Variation Choices window, you can add these kinds of very specific restrictions. An unselected button means that given a matching set of chord conditions, this variation will not be chosen. Unlike with the 32 variation buttons, the button pressed/ unpressed state is actually used here to determine in-application behavior. However, as with the variation buttons themselves, a disabled variation's row will be grayed out.
The Functions area allows you to specify specific chords that a variation should or should not be played over. The capitalized version of the chord (e.g., I) indicates the major version, the lowercase version (e.g., i) is minor, and italicized (e.g., i) is for all other chord types (augmented, diminished, etc.). The Root column allows you to specify whether a variation should play over notes that are based within the scale (S), or based on a non-scalar tone that is flatted (b) or sharped (#). Type allows you to determine whether the variation should play over a triadic chord (tri), a chord with a sixth or seventh (6, 7), or more complex chords (Com). Lastly, Dest, or destination, actually looks ahead to what the next chord in the progression is going to be. If the current chord is leading to the tonic (->I), a dominant chord (->V), or any other chord (->Oth), you can instruct a variation to never play in one or more of these situations.
Look at Figure 3-15. Assuming we're in the key of C, this variation will only play if the chord is C major (I), d minor (ii), F major (IV), G major (V), or a minor (vi). It will also only play over scalar roots, so, for instance, it would not play over Ab minor. The variation will play over any chord "type" (triadic, 6th/7th, or complex). Lastly, this variation will only be chosen if the next chord encountered in the Segment is a C major (I) or G major (V) chord.
One possible configuration for a variation.
Audio producers and programmers often ask if there is a way to force a specific variation choice. The Variation Choices window provides one option for doing this. Without using any of the standard chord functionality (notes being revoiced and/or transposed to fit the chord), you could make every variation play over a unique chord and then choose the particular chord to force that single "legal" variation to be chosen.
Interactive and Adaptive Audio
Scott Selfon
So far, we have covered playing back basic linear audio and adding aspects of variability to that music. The next step is to make the audio dynamic — that is, allow it to respond to events that the user or program creates (i.e., interactive and adaptive audio). For instance, computer games often want to convey a certain emotional level based on what the current game state is. If the player's health is low and many enemies are around, the composer would often like the music to be more frantic. This general concept of an "intensity level" is often difficult to communicate to the programmer and even more challenging to implement convincingly in an application.
Variable Intensity Music and Ambience
DirectMusic provides several potential solutions for creating dynamic music. In this chapter, we cover two in particular — the use of DirectMusic Style files and the use of Segment-based branching.
DirectMusic Style Files
Perhaps the easiest way to implement intensity level is to use DirectMusic Style (.sty) files, which contain numerous musical fragments, each with an assigned intensity level. Audio producers compose one or more patterns (musical loops) within a Style. A pattern is composed of a Pattern Track, those same MIDI-supporting Tracks we've already used in Segments. Each pattern is assigned a groove level range, DirectMusic's term for musical intensity. When the programmer alters the groove level based on application events, a new pattern is chosen automatically based on the new intensity requested.
When a new Style file is created (File>New…, or the Ctrl+N shortcut), it appears in the project tree with several folders.
Figure 4-1: A newly created style file. Remember that design-time file extensions end with p, so this is a .stp file. The run-time version of the file will be a .sty file.
The Bands folder contains one or more Bands (information about per-channel volume, pan, and patch change settings) that can be dragged into a Segment's Band Track. Typically, you will just use one Band per Style, and that Band is automatically inserted when you drop the Style into a Segment. More information on that in a bit.
Motifs provide a subset of the functionality available to secondary Segments (and for history buffs out there, they are actually the predecessor to secondary Segments). Recall that one or more secondary Segments can be played at a time and are typically layered on top of a primary Segment, whereas only a single primary Segment can be playing at a time. While motifs may appear somewhat convenient (all living in a single file), there are several significant drawbacks to motifs:
They cannot be accessed directly using DirectX Audio Scripting. DirectX Audio scripts can only play and stop Segments, not motifs (which reside in Styles).
While they can be played directly by a programmer, the composer needs to manually embed a Band within the motif to ensure that its expected instrument patch changes are applied before it is played.
While motifs support variations, it's up to the game programmer to keep using the same instance of the motif in order to avoid resetting the variation order.
In short, for most cases, secondary Segments are much more useful and convenient than motifs.
So we're typically going to have a Style with a single Band, no motifs, and several patterns. Our default Style has a single pattern. If we double-click it, the Pattern Editor (looking quite similar to the Segment Designer window) opens. MIDI controller editing is identical to the behavior in Pattern Tracks in Segments.
Figure 4-2: The Pattern Editor window.
There is a new Track at the top of the part called the Chords for Composition Track. This Track, covered in more detail in Chapter 6, tells DirectMusic what the original function of the music was so that DirectMusic can make an educated choice when revoicing the music to work over a new chord or key. For now, we leave it at the C major default.
Note DirectMusic Style files only support Pattern Tracks and the Chords for Composition Track and therefore DLS Instruments. In particular, Styles cannot be used in conjunction with Wave Tracks, so consider Segment-based branching if you are using prerendered and/or streamed music.
Opening the properties page for a pattern gives us access to groove range and other pattern-related functions. This is also where you can adjust the length of this specific pattern.
Figure 4-3: The properties page for a pattern within a Style.
We've discussed groove range earlier. An audio producer can assign a pattern to a single groove level if they wish or a range of appropriate groove levels. Typically, a range of values is used. This allows for two things:
Pattern variability. If two patterns have overlapping ranges, a requested groove level within the overlap will randomly choose between the two patterns (all other aspects of the patterns being equal — see below for more details on how a pattern is chosen).
Easy addition of patterns later. Let's say the composer creates three patterns — a low-intensity pattern (assigned to groove range 0–25), a medium-intensity pattern (assigned to groove range 26–75), and a high-intensity pattern (assigned to groove range 76–100). After trying the application, the composer might decide another pattern with intensity between "medium" and "high" is appropriate. He can create a new pattern without the programmer needing to make any changes to his code. By contrast, using a single groove level, rather than a range, for each pattern would probably require the programmer to change how he determines the groove level.
The Destination Range option can only be used when you know what groove level your music is heading toward — that is, generally only when groove levels are authored directly into a Segment via a Groove Track (see the "Using Styles with Segments" section later in this chapter). With destination range, DirectMusic will actually "look ahead" to what groove level is going to be needed next before choosing which pattern should be played. While this could be somewhat useful when "ramping" from one groove level to another, the idea of authoring groove levels into content is typically contrary to the desire for dynamic manipulation of groove levels. For this reason, destination range is rarely used.
The Embellishment area allows you to add even further control to how a pattern is chosen. Embellishments are generally specific patterns that you can call for without changing groove levels. For instance, you might be playing a piece of music at groove level 50 and you want to transition to a new piece of music, also at groove level 50. Rather than temporarily changing the groove level to pick a convincing ending pattern, you could create a pattern still assigned to groove level 50 that is an end embellishment. To further strengthen the flow of the transition, the new Style could have an intro embellishment pattern. Then when the programmer or scripter wants to start playing the second piece of music, he can say to play ending and introductory embellishments, and these specially assigned patterns will be played between the two pieces of music. The naming conventions for embellishments (intro, end, fill, break, and custom) are actually somewhat arbitrary — from a programmer's or scripter's point of view, there's no difference between most of them. The exceptions are that an intro embellishment is always picked from the Style you are going to, and the end embellishment is always picked from the Style that you are coming from. Of course, if you're sticking with the same Style and just wanted a brief pattern change, even these have no difference in function. Note that custom embellishment types (with a range from 100–199 to help keep them from becoming confused with groove ranges) can only be requested programmatically or within content; DirectX Audio Scripting does not support them.
The Chord Rhythm option adds yet another layer of control over how DirectMusic chooses which pattern to play, assuming the Segment uses chords (in Chord Tracks or via ChordMaps). The audio producer might have one pattern that is more ideal to a rapidly changing progression and another that works better under a static chord. DirectMusic will look for a pattern with a chord change rhythm that maps closest to the currently playing Segment's chord rhythm.
Choosing a Pattern from a Style
So after all of these settings, how exactly does DirectMusic choose which pattern to play at a given time? In descending order of priority:
Look for the embellishment requested.
Look for the groove level requested.
Look for pattern matching destination groove level to next groove change. Again, as destination groove level is rarely known ahead of time, this particular rule is rarely used.
Longest pattern that fits available space (between now and end of Segment or next groove change).
Look for a pattern with an appropriate chord rhythm.
In case of a match in all other settings, randomly choose one of these matching patterns to play.
The fourth rule is often the most significant to keep in mind. The argument here is that DirectMusic should pick the musically "strongest" progression, which would mean to prefer to play a single eight-bar pattern rather than a one-bar pattern eight times. But the rule also applies when you have multiple longer patterns to choose between. If you had been anticipating using variable length patterns, it's generally not easy to get around this restriction but it does allow for a simpler composition and musical transition process.
Building Up Styles
The most basic use for Styles is what we like to call additive instrumentation. Each pattern is authored identically to the previous (lower groove level) one but adds another instrument to the mix. This can work quite effectively and is relatively easy to implement. The audio producer creates a low groove level pattern (which could be silence, basic rhythm, or perhaps just subtle ambience). They then create additional patterns, each at a higher groove level and each adding a new instrumental line or additional rhythmic complexity.
Primarily due to the pattern-choosing rules discussed above, it is often easiest to create every basic pattern with the same length. Embellishment patterns of differing lengths can help break up the rhythm somewhat, though all patterns must use the Style's global time signature for their playback length. Individual parts in a pattern can be varying lengths and time signatures to help distract from the feeling of constant time signature.
By the way, groove level and pattern length can be viewed in a somewhat easier-to-read interface via the Style Designer window. The window is opened by double-clicking on the Style itself in the project tree rather than any of its patterns and displays information on all of the patterns (their length, groove ranges, embellishments they are assigned to, and so on).
Figure 4-4: The Style Designer window for a Style with five patterns for varying groove levels. The highlighted pattern is an ending embellishment (as indicated by the E in the Embellishment column).
Using Styles with Segments
Remember that the basic unit of playback (as far as the programmer is concerned) is typically the DirectMusic Segment, not a Style file. So we're going to want to create a Segment that actually "plays" our style. Several Track types come into play when instructing a Segment on how to use one or more Styles:
Style Track: This Track is the basis for using Styles with Segments. Typically, the Style Track will consist of a single Style placed in the first bar. However, there's nothing wrong with changing Styles as a Segment proceeds.
Groove Track: This is the Track where the audio producer can set the current groove level for the Segment. This Track also allows for a Segment to manually request an embellishment pattern (for instance, play an intro embellishment at the beginning of the Segment). Inserting a Style into a Style Track automatically makes changes to several other Tracks in a Segment, inserting new Tracks if necessary.
Time Signature Track: This Track becomes locked to the time signature of the current Style, as you can't override a Style's time signature in a Segment. If additional Styles are inserted later in the Segment, similarly locked time signature events (which appear "grayed out") will appear.
Tempo Track: The default tempo for the Style (specified in the Style's property page and the Style Designer window) is inserted on the first beat of the bar where the Style was entered.
Band Track: The Style's default Band is inserted one tick before the bar where the Style begins playback. This ensures that all of the instrument changes and volume/pan settings are applied just before the first notes are heard (assuming no notes begin before beat one — if any notes do occupy the pickup bar, you should manually move the Band even earlier). Note that the Band inserted into the Band Track is a copy of the Style's Band, not a "reference" to it. That is, if you now go back and edit the Style's Band, those changes are not applied to the Band in the Segment. Similarly, if you edit the Band in the Segment, those changes won't be reflected in the original Style's Band. We typically manually drag the Style's Band into the Segment just to make sure we are not using an old version of the Band.
So all of these Tracks are automatically filled in for us. But what about the Groove Track? Because the groove level is something we want to be dynamically changed by the programmer while the application is running, composers don't typically author groove changes into the Groove Track beyond an initial setting (typically the lowest groove level) in the first bar.
Figure 4-5: A typical Style-based Segment. Groove level 1 is specified in the first bar. When the Style was inserted, the other tracks were created and/or modified automatically.
Before we completely dismiss the Groove Track, it is still quite useful for auditioning groove changes within DirectMusic Producer. After composing a bunch of patterns of varying groove levels, you might create a long Segment, drop in the Style and a few groove changes, and listen to how the performance sounds. You could even use variable groove changes (the +/– option in a groove change's property page) or experiment with the embellishment types.
As far as the length of the Segment, you have several options. If you are using patterns of varying lengths, you'll generally want to make the Segment the least common multiple of those lengths. For instance, if one pattern is four bars and another is three, you'll want to make the Segment 12 bars long (and infinitely looping). This just makes sure that, assuming a single groove level (and often a single pattern) is played for a long time, the pattern won't be interrupted or cut short by several bars due to the Segment ending. Worse, remember that the pattern chosen is also influenced by how much space is left before the end of the Segment — so you might get a completely different pattern chosen right at the end if the pattern length did not match the Segment length.
For the specialized (and easier to manage) case where all patterns have exactly the same length, creating an infinitely looping Segment of that same length is generally preferable. This will also give us some added flexibility in how transitions between the groove levels occur, particularly with the ability to successfully use alignment transitions.
Changing the Playing Patterns
If you've experimented with groove changes to audition your patterns in a Segment, you might notice that the new groove level's pattern didn't always take over immediately. The reason behind this is that a groove change tells DirectMusic to choose a more appropriate pattern when the currently playing pattern finishes. Certainly, the application might change groove levels quite a bit, and you wouldn't want new patterns abruptly starting and restarting frequently. However, if you've created a lot of 20-bar patterns, the change to the new groove level could definitely take quite a while and more time than you would hope.
There are several options for making pattern changes occur more frequently. The easiest will involve the programmer or scripter. Every time he adjusts the groove level, he can also restart the currently playing Segment using any of the transition types we've already discussed. For instance, the audio producer could instruct the programmer to restart the Segment at the next bar. This means that a groove change will "take effect" (a new pattern will begin playing) within the next bar. Or, in the above example, the audio producer could structure his 20-bar patterns such that they're actually composed of five four-bar-long sections. The audio producer could then put a Marker Track in the Segment (note that the markers are not in the Style file, as Style patterns don't support Marker Tracks) at the natural break points for the music, and the programmer could restart the Segment on a marker boundary. One difficulty with this solution is that we are restarting the Segment (and thus whatever the appropriate pattern is) every time we touch the groove level. This can be problematic when smaller groove level changes shouldn't really affect the pattern we're playing. For instance, if our "low level" pattern spans groove levels one through 25, it doesn't really matter to us if the groove level changes from one to two. The same piece of music should just keep playing. But in the simplest implementation of the above solution, we're retriggering the Segment — and thus restarting the pattern — every time the groove level changes. Depending on how frequently these changes occur, we could get very intimately acquainted with the first few bars of the pattern and never hear the 20th onward. One way to combat this is for the content creator to instruct the programmer as to when the Segment should really be restarted versus allowing it to continue with the newly adjusted groove level.
Based on the above issue with potentially restarting the playing pattern from the beginning constantly, another commonly used technique is to use alignment to Segment transitions. Remember, in this kind of transition, we jump to the same relative location in the destination Segment as we were at in the previous Segment. So if we were at bar four beat two, we jump to bar four beat two of the new piece of music. In the particular case of adjusting groove levels, both the source and destination Segment are the same piece of music. The retriggering is enough to bump us over to a new pattern more appropriate for the current groove level. But with the use of alignment, we transition over in mid-pattern rather than restarting the Segment. This technique is a bit trickier to manage than the previous one. Remember that if we're starting in the middle of a pattern, any notes that would have started playing prior to that point would not be retriggered — so for instance, if our destination had some nice string pads that had started on the downbeat, we wouldn't hear them if our transition occurred on beat two. An added complexity occurs if our patterns had different lengths, as they now may transition to musically awkward positions in each other. This is another reason why it's often easiest to keep all patterns at the same length.
Figure 4-6: Attempting to transition using Segment alignment can be difficult for patterns of differing lengths. In the above diagram, Pattern 1 is four bars long, and Pattern 2 is five bars long. Below them is a display of the 20 bars of a looping Segment, so we can see how they line up over time. If we transition from Pattern 1 to Pattern 2 at, say, the end of bar eight, we're transitioning at the end of Pattern 1 but into the start of the fourth bar of Pattern 2. If both patterns had been the same length, we would be making a potentially more natural transition from the end of Pattern 1 to the beginning of Pattern 2.
Segment-Based Branching and Layering
An alternative to DirectMusic Style files for creating variable intensity music and ambience is to use multiple Segments. Remember that with strictly Style-based playback, you are restricted to DLS instruments. With Segments, you have the ability to mix and match between DLS instruments and prerendered streaming wave files. There are two primary ways that Segments can be used to shift intensity: branching and layering.
Branching Playback
In the case of branching Segments, we change the Segment we are playing at a musically appropriate boundary, based on some event that changes the emotional level we want to convey. In the simplest implementation, branching just consists of authoring complete pieces of linear music for each emotional intensity, and then picking logical places where one piece could jump to the beginning of the next. Often measures are appropriate boundaries; if not every measure is, the composer can use markers to dictate which are.
Sometimes the above can result in abrupt and unsatisfying transitions, particularly if the two pieces of music were not written with each other in mind. The jolt from playing a slow-moving piece of music to a pulse-poundingly fast one might be appropriate in some circumstances, but in others it can distract more than intended. A potential solution here is to use transition Segments.Aswepreviously discussed, when you tell a new Segment to start playing, you can specify a transition Segment that should be played before this new Segment starts. A common solution to the abruptness issue is to create quick little "bridges" that ramp up or down the tempo, smooth out any instrument entrances and exits, and so on. Of course, this can lead to a bit of extra work on the composer's part — you might find yourself writing transition Segments for every possible configuration (slow to fast, fast to slow, slow piece 1 to slow piece 2, slow piece 2 to slow piece 1, and so on) unless you're clear as to what pieces could possibly jump to other pieces. Building up a roadmap beforehand can aid both in the planning and authoring of transition Segments.
Figure 4-7: A sample diagram for five pieces of music. Each arrow indicates that a piece of music will need to be able to transition to another piece of music. For this scenario, we assume that you have to somehow "engage" enemies in combat, so you'll never jump right to the "fighting" music from the "no enemies" music. Similarly, if you decide to flee the room, you'll disengage from the enemies first, so you don't have to worry about the "fighting" music jumping directly to the hallway "ambient" music.
Yet another alternative use of branching is Segment alignment. Remember that by default, a new Segment will start playing from the beginning. Using alignment, we can make it start at the same position where the last Segment left off. With this technique, we can achieve results similar to the Style-based playback above, but with the benefit of streamed waves. Using this technique, you could author several different versions of the same audio track, with different mixes and increased instrumentation in each version. Again, using markers (in marker tracks) to control what are appropriate boundaries, the music can seamlessly ramp up or down to another wave based on the scenario.
Layered Segments
The other effective Segment-based tool for creating dynamic music is layering. Here we keep playing the same basic piece of music, but we layer other pieces of music over the top. This involves the use of secondary Segments. Remember that only one primary Segment can be playing at a time, and playing a new primary Segment automatically stops any currently playing ones. But you can layer as many secondary Segments on top of a primary Segment as you like. Don't forget that whether a Segment is played as primary or secondary is in the hands of the programmer or scripter, so the composer needs to communicate this information beforehand.
A common use of layered secondary Segments is as motific elements. Indeed, these secondary Segments replace the somewhat outdated Style-based motifs, which were much less flexible and required extra effort from the programmer or scripter in order to use them. As an example, let's say our character's love interest enters the room, and the love theme plays on top of the basic music track. The use of Chord Tracks (described in more detail in Chapter 6) can make this all the more effective. The love theme can be played in the appropriate key, and even repitch itself based on the chord progression of the underlying music.
As a brief example, let's take any old Segment we've already authored and add a Chord Track to it if it doesn't already have one. (Right-click on Segment > Add Track(s)…, and choose Chord Track from the list.) Now let's right-click on the Chord Track (the leftmost part of the track that says "Chords") and bring up its properties. Odds are the track thinks our piece of music is in the key of C because we haven't told it otherwise. Mine happens to be in Eb, so I update the property page like so:
Chord Track Properties dialog.
We'd probably also want to adjust that C major 7 chord that appears in measure 1 by default. Let's make it an Eb major 7 instead. (I'm not going to bother entering the full chord progression, but we'll use this particular chord in a moment.) We right-click on the CM7, choose Properties, and modify the property page like so:
Our Eb major seventh chord. Adjusting layer 1 (the bottommost chord layer) defaults to automatically editing all four layers, so you would only need to adjust the chord (four-octave keyboard on the left) and the scale (one-octave keyboard on the right) once. Right-click and uncheck Auto-Sync Level 1 to All to change this behavior.
Now let's create our romantic theme. We create a new Segment with a Pattern Track and a Band Track (making sure that the Band and pattern pchannels don't interfere with any from our original piece of music). As far as the actual music, I'll just borrow a recognizable theme from Tchaikovsky's Romeo and Juliet:
Our (well, Tchaikovsky's) love theme. Note that we've authored it in the key of C (not having a Chord Track implies this) and named the Segment "LoveTheme." The Pattern Track part uses performance channel 17 (which can be adjusted from the property page), and we disabled all variations except our theme on variation 1. Our band similarly just has an entry for performance channel 17 telling it what instrument to play.
If we played our theme as a primary Segment, it would play in C major. But if we play it as a secondary Segment against our piece of music in the key of Eb, it will transpose to this key. To try it out, open up the Secondary Segment Toolbar (from the View menu, choose Secondary Segment Toolbar), and choose our Segment from one of the drop-downs.
The Secondary Segment toolbar.
You can specify the boundary it will audition on by right-clicking the Play button. Now play the primary Segment from the main transport controls (click on the Segment to give it "focus," which will make its name appear next to the primary segment's Play button). As it plays, play our love theme from the secondary Segment toolbar's Play button. The secondary Segment transposes to fit the key of our primary Segment. Similarly, this theme could be played against any of our linear compositions in the proper key just by authoring a Chord Track into those Segments and specifying the proper key and scale in a chord in the track.
Transposition is one thing, but as a more advanced use, we might want our theme to actually follow the chord progression of our primary Segment. Let's add another chord to our primary Segment and start off with a dominant-tonic (V-I) progression. We'll move that Eb chord to bar two (by dragging or copying and pasting) and insert a Bb chord in bar 1.
Now when we play our secondary Segment as this primary Segment plays, the notes will conform to the current chord as well as the scale. We can adjust the specifics of how our secondary Segment adjusts itself from its property page, and can even override these settings on a per-note basis if we wish.
The Default Play Mode area determines how our secondary Segment responds to chord changes. In the default behavior, chord and scale are used to determine appropriate notes, but there are also settings to only transpose according to scale, to remain fixed to the authored notes (useful for drum tracks), and so on.
Alternatively, secondary Segments can be played as controlling secondary Segments. In this case, rather than taking on the characteristics of the primary Segment, they force the primary Segment to take on their characteristics of tempo, chord progression, and so on — whichever tracks are present in the secondary Segment are used to replace to the primary Segment's. Going back to our previous example, we can add a Chord Track to our secondary Segment telling DirectMusic that our theme was indeed authored in the key of C, overaCmajor chord. If we then played it as a controlling secondary Segment over the primary Segment, the primary Segment would now take on the chord progression of the secondary Segment.
Adding a new Chord Track and C major chord to our secondary Segment.
Setting up a Play button in the secondary Segment toolbar (by right-clicking on it) to play a controlling secondary Segment. Don't forget that this is an audition setting only; the composer would want to tell the programmer or scripter that the Segment should be played as a controlling secondary Segment.
Remember that Sequence Tracks do not respond to chord changes — they are static, non-pitch-varying tracks, analogous to traditional sequenced MIDI tracks. So the fact that our controlling secondary Segment has switched out the chord progression won't be heard in the primary Segment unless any of its notes are played by Pattern Tracks. But if our music was placed in a Pattern Track, and we had traced out the chord progression for the music in our Chord Track, the primary Segment would sound quite different over this new chord progression. To repeat a useful scenario for this, background music authored in a major key could switch to a minor key when the villain enters the room, just by playing a controlling secondary Segment composed of minor chords. Indeed, there is nothing that says a secondary Segment has to be composed of any notes at all — it is perfectly acceptable to use them for Tempo, Chord, Style, Wave, and any other tracks.
In this chapter, we have attempted to supply an overview and a basic understanding of some of the power and flexibility DirectMusic can bring to an interactive application. It can be used simply to provide a MIDI-capable synthesizer, with the ability to deliver instrument collections that ensure uniform content playback across multiple sound cards. It can be used to synchronously play back real-time-rendered (MIDI) and prerendered (wave) sounds in synch with each other. DirectMusic provides the ability to seamlessly transition from one piece of music to another at appropriate (and content-defined) boundaries. And it gives a rich set of tools to composers and sound designers for creating variation and dynamically adjustable content. Being aware of some of the rules and restrictions for how it can best be used, I encourage you to experiment with the concepts presented here. It's often easiest to gain familiarity with a few tools at a time rather than being overwhelmed by the options; this chapter has merely touched the surface of the features DirectMusic supports.
Okay, we understand linear pieces of music and getting from one piece of music to another. A primary challenge to overcome is repetition. No matter how smooth our transitions, if we hear the same pieces of audio repeatedly, they are bound to get stale. Here is where DirectMusic provides significant strength!
Variable Playback/Variations
Content variability can be achieved on several different levels. On a "microscopic" level, content can specify ranges for volume and pitch on individual waves played in Wave Tracks, as well as randomized volume, start time, and duration on notes in Pattern Tracks.
The Wave Properties page allows you to specify ranges for attenuation and fine-tuning. This particular example will randomly pitch-shift the wave up or down by two semitones.
The Pattern Note Properties page allows you to specify variability for velocity, start time, and duration. This particular note will play with randomization in start time, duration, and with a random velocity between 75 and 125.
On the "macroscopic" level, you could of course author multiple Segments and write a script to choose between them randomly.
Then we have variations, which provide the best of both worlds. Variations are those 32 intimidating-looking buttons present on Wave Track parts and Pattern Track parts (in Segment files), as well as in pattern parts (in Style files, a format which we cover in a bit).
Each of those buttons represents one version of the part; for instance, you could have a drum part where variation 1 includes the full drum part, variation 2 removes the snare, variation 3 mixes up the rhythms a bit, and so on.
The primary thing to remember about variations is that at any given time, a single variation plays per part. This does not mean that every single part is playing variation 12 at the same time, but it does mean that if you author 12 variations for your drum Track, only one is being played at any given time. Each part chooses a new variation to play whenever its Segment is started or loops.
Before we get into the details of variation behavior and functionality, let's talk a bit about the 32 buttons, which can be among the trickier aspects of DirectMusic Producer to understand. When you first create an empty part that supports variations, each of the 32 button labels is white. This indicates that these variations are empty. As soon as you add any note, continuous controller, or wave information to a variation, that variation's label becomes black, which indicates that it has some information in it.
One potentially confusing aspect of variations is that empty variations are just as acceptable to be "picked" to be played as variations with data. For instance, a composer might create three alternate versions of a drum part on variations 1 through 3. When he then tries to play the piece of music, he finds that most of the time it's silent. Why? Because variations 4 through 32 were enabled and empty (silent), meaning 29 out of 32 times, an empty variation was chosen.
This brings us to disabling variations. When a variation is disabled, DirectMusic will never "pick" it to be played. By default, all variations are enabled. This is indicated in DirectMusic Producer by a solid gray background for the button.
which typically should be deleted (since the variation will never be chosen).
To disable one or more variations, make sure the appropriate variation is pressed by clicking on it; then right-click anywhere over the 32 variation buttons and choose Disable from the right-click menu that appears. Disabled variations have a checkerboard look to them. Note that it is possible to have a non-empty variation that is disabled. In most typical scenarios, this means that you are saving data that you will never use, so it is generally a good idea to go ahead and erase any disabled variations when your project is completed.
Auditioning Variations in DirectMusic Producer
The final aspect of the user interface for variations is the buttons being selected ("pressed down") or not selected ("up").
The main thing to remember about this aspect of the buttons is that it is solely for editing and auditioning purposes in DirectMusic Producer — whether a button is pressed or not has no bearing on how the variations will behave in a DirectMusic player or in a game at run time. Pressing buttons provides for two purposes:
Editing one or more variations simultaneously. Pressing a single variation button means that you are now editing only that variation. Paste from the clipboard, add a note, or drag a wave into the part, and that data is assigned solely to that variation. If you select multiple variations, notes and waves added will be identically added to each of the selected variations.
Auditioning variations within DirectMusic Producer. Let's say you created a four-part piece of music, and each part had ten enabled variations. You listen to it a few times and notice that there's an incorrect note in one variation of the bass part. Rather than forcing you to keep repeatedly playing the Segment until that variation gets picked again, DirectMusic Producer has an audition mode, where the same variation (whichever one is "pressed") will be chosen every time the Segment is played. Only one part can be in audition mode at a time. To select a part for audition mode, just click anywhere on it. The part that is in audition mode is highlighted in a slight yellow shade.
Part three (piano) is in audition mode. Within DirectMusic Producer, all other parts will randomly choose which variation to play from the enabled variation buttons. Part three will always play whatever variation button is pressed (until it is no longer in audition mode).
Remember that audition mode only applies in DirectMusic Producer; when a Segment is saved and played in a DirectMusic application, none of its parts will be in audition mode, and all parts will choose from their enabled variations.
Audition mode provides for some unique behaviors not seen outside of DirectMusic Producer. First, as multiple variation buttons can be selected ("pressed") at once, you have the opportunity to hear a single part playing multiple variations at once. As mentioned previously, only a single variation will be chosen at a time in an actual DirectMusic application. You can also select disabled variations and audition them; a DirectMusic application will never play a disabled variation. Lastly, you can have all of the variations on your auditioned part deselected, in which case no variation will be chosen, causing this part to be silent. Again, this behavior is solely an audition feature of DirectMusic Producer.
Variation Shortcuts
There are a few shortcuts provided for quick selection of which particular variations you are editing. The vertical bar to the left of the 32 variation buttons selects (and with a second click deselects) all enabled variations. Double-clicking on a variation will "solo" it (only select that variation). Double-clicking again will "unsolo" it, reverting to the previously pressed buttons.
If you are using linear wave files, a shortcut is provided to avoid needing to work directly with the 32 variation buttons at all. From the File menu, choose Import>Wave File(s) as Segment Variation(s)…. From the dialog that appears, choose all of the wave files that you want to be variations in your Segment. A clock-time Segment is automatically generated, with a length of the longest wave file you chose and using one variation per wave file. You'll notice that in this case, all of the empty variations (unless of course no variations are empty) are automatically disabled.
Specifying Variation Behavior Settings
Once you have created a Segment that utilizes variations, the next logical step is to be able to more finely control how variations are chosen. For instance, if we had 20 variations of a dialog fragment, we would really like the person hearing this Segment not to hear the same variation played twice in a row. Actually, it would often be ideal to play every possible variation once before we ever get a repeat. This kind of functionality can be specified by the audio producer from each part's properties page.
As usual, right-click a part and choose Properties (or if the Properties page was already open, click the part to make it the active property display).
Figure 3-8: Selecting per-part variation behavior for a Pattern Track part.
Figure 3-9: Selecting per-part variation behavior for a Wave Track part.
Though presented in different locations on the Pattern Track and Wave Track property pages, the available options and functionality are the same:
In random order: The default behavior — a new variation is randomly selected from the enabled variations every time the piece of music plays or loops. Repeats can occur.
In order from first: The first time the Segment is played, variation 1 (or the first enabled variation) is played. On each repeat or loop, the next enabled variation in order is played.
In order from random: The first time the Segment is played, a random enabled variation is chosen. On each repeat or loop, the next enabled variation in order is played.
With no repeats: Random enabled variations are chosen on each repeat or loop, but the same variation will not be chosen twice in a row.
By shuffling: On each repeat or loop, a random enabled variation is chosen, but every enabled variation will be played once before any are repeated.
Recall that a new variation is chosen each time a Segment is played or it loops. However, there are situations where we might want to "reset" the variations. For instance, let's say that each variation is the next step up in an ascending scale, and we allow the Segment to loop as the character picks up tokens. If they haven't picked up any tokens in the last few minutes, we want to stop the Segment and start the scale over from the lowest note the next time a new token is picked up. This is where the Reset Variation Order on Play option (available in The Wave Track and Pattern Track property pages) can be used to control whether DirectMusic "remembers" which variations were previously played the next time the Segment starts. If, for instance, a Segment was set at In Order from First and its Pattern Track had Reset Variation Order on Play checked, every time the Segment was played, it would start from variation 1. If it looped, it would then play variation 2, 3, and so on.
Reset Variation Order on Play functionality for the Pattern Track.
There are, of course, instances where the audio producer would prefer for two or more parts to always pick their variations identically. For instance, in a DirectMusic arrangement of a jazz trio, there might be several "solos" for each of the instruments (parts). We do not want two instruments to "compete" by playing their solos at the same time. For these kinds of situations, Lock ID functionality provides a way to control how different parts choose their variations. Every part that has the same Lock ID will choose its variation to be the same. Therefore, for the above scenario, if each part has three solos to choose from, we could place the drum part's solos on variations 1-3, the guitar's solos on variations 4-6, and the piano's solos on variations 7-9, for example. For the variations where the parts do not have solos, they would either "lay out" or just keep time. When the Segment was played, each of the parts would choose its variations in step with the other parts, so only one solo would be played at a time.
This Wave Track part's Lock ID is set to 1. Any other Wave Track parts in the Segment that are set to Lock ID 1 will choose the same variation as this part.
Note that Lock ID does respond to the other settings that we've discussed that control how variations are chosen; if one locked part has variation 10 disabled, that variation will never be chosen for that entire Lock ID group. Similarly, if one part is set to play its variations in order, all of the parts will follow that part's lead. Mixing and matching between various part variation settings (for instance, one part set to shuffle, another part set to play in order) within a Lock ID group can lead to somewhat unpredictable variation choices, so it's generally recommended that all parts in a Lock ID group use the same settings.
Another important concept to remember is that Lock ID is unique per Track, not per Segment. That is, several parts in a Pattern Track that use the same Lock ID will choose the same variation. However, a Pattern Track part and a Wave Track part using the same Lock ID will not lock to each other. A limited subset of this kind of functionality is available in the Wave Track's property page with the Lock Variations to Pattern Track option. Checking this box means that each pchannel of the Wave Track is now locked to the corresponding pchannel of the Pattern Track; a Wave Track part on pchannel 1 will always choose the same variation as the Pattern Track part on pchannel 1 if one is present.
Locking a Wave Track's variation to a Pattern Track.
Locking a Wave Track's variations to a Pattern Track is most useful when you wish to use MIDI continuous controllers to manipulate waves — panning, MIDI volume control (fade in, fade out, etc.), and so on. By authoring corresponding variations in the Pattern Track, you can ensure that a particular wave plays using particular MIDI controller information. Alternatively, you could keep the Wave Track unlocked from the Pattern Track, in which case any of your controller variations might be used over any Wave Track variation.
Variation Switch Points
As we've already discussed, a new variation is chosen for each part whenever that part's Segment starts playing or loops. But what if we want variations to be chosen more frequently? In particular, the audio producer might create small fragments of melodic or rhythmic lines that he wants to be able to switch between in the midst of a Segment. Variation switch points provide the audio producer with a solution to this — namely, the ability to determine if and when a part can change variations in the middle of playback.
Variation switch points are displayed in DirectMusic Producer in a separate strip for each Pattern Track part (they are not supported by Wave Track parts). The two types of switch points are enter switch points and exit switch points. Enter switch points tell DirectMusic where it is legal to "jump into" this particular variation. Exit switch points tell DirectMusic where it is legal to try to find another variation to change over to. If no variation has an enter switch point corresponding to this exit switch point, the currently playing variation will continue to play.
A variation using variation switch points. This variation can jump to any other variation at the position of the exit switch point (red blocks), shown as the dark blocks in the Var Switch Points line. Similarly, this variation can start playing mid-Segment if any other variation has an exit switch point corresponding to this variation's enter switch point (green blocks), the lighter blocks in the Var Switch Points line.
While opening many possibilities for potential variation mixing and matching combinations, variation switch points can make composition much more challenging. Rather than only having to worry about how different variations meld at Segment loop points, variation switch points open up the possibility of playback jumping between variations right in the middle of playback, at whatever boundaries they wish. Often it is easiest to limit variation switch points to measure boundaries, although they do support down to grid resolution if you need it.
Influencing Variation Choices
DirectMusic provides even more acute functionality for controlling which variations play when, particularly when the audio producer is using chord progressions. We cover the various aspects of chord proressions (chords, Chord Tracks, and ChordMaps) in more detail in Chapter 6, but for now just know that a composer can define a static chord progression by placing chords in a Chord Track or a dynamically generated progression by using a ChordMap and a Signpost Track. Regardless of how the chords generate, a composer can elect to have variation choice influenced by the function of the currently requested chord in the current key. This functionality is in the Variation Choices window, more informally known as "The Mother of All Windows." The window opens from the button to the right of the 32 variation buttons:
Figure 3-14: The Variation Choices window.
In this window, each row corresponds to one of the 32 variations. Each column allows the audio producer to specify exactly what kinds of chords a variation will or will not be chosen for. For instance, you might have a melodic line in a variation that sounds great played over a IV chord (e.g., an F major chord in the key of C), but you have another variation that provides a stronger option to play on a ii chord (e.g., a D minor chord in the key of C). By selecting and unselecting the various buttons in the Variation Choices window, you can add these kinds of very specific restrictions. An unselected button means that given a matching set of chord conditions, this variation will not be chosen. Unlike with the 32 variation buttons, the button pressed/ unpressed state is actually used here to determine in-application behavior. However, as with the variation buttons themselves, a disabled variation's row will be grayed out.
The Functions area allows you to specify specific chords that a variation should or should not be played over. The capitalized version of the chord (e.g., I) indicates the major version, the lowercase version (e.g., i) is minor, and italicized (e.g., i) is for all other chord types (augmented, diminished, etc.). The Root column allows you to specify whether a variation should play over notes that are based within the scale (S), or based on a non-scalar tone that is flatted (b) or sharped (#). Type allows you to determine whether the variation should play over a triadic chord (tri), a chord with a sixth or seventh (6, 7), or more complex chords (Com). Lastly, Dest, or destination, actually looks ahead to what the next chord in the progression is going to be. If the current chord is leading to the tonic (->I), a dominant chord (->V), or any other chord (->Oth), you can instruct a variation to never play in one or more of these situations.
Look at Figure 3-15. Assuming we're in the key of C, this variation will only play if the chord is C major (I), d minor (ii), F major (IV), G major (V), or a minor (vi). It will also only play over scalar roots, so, for instance, it would not play over Ab minor. The variation will play over any chord "type" (triadic, 6th/7th, or complex). Lastly, this variation will only be chosen if the next chord encountered in the Segment is a C major (I) or G major (V) chord.
One possible configuration for a variation.
Audio producers and programmers often ask if there is a way to force a specific variation choice. The Variation Choices window provides one option for doing this. Without using any of the standard chord functionality (notes being revoiced and/or transposed to fit the chord), you could make every variation play over a unique chord and then choose the particular chord to force that single "legal" variation to be chosen.
Interactive and Adaptive Audio
Scott Selfon
So far, we have covered playing back basic linear audio and adding aspects of variability to that music. The next step is to make the audio dynamic — that is, allow it to respond to events that the user or program creates (i.e., interactive and adaptive audio). For instance, computer games often want to convey a certain emotional level based on what the current game state is. If the player's health is low and many enemies are around, the composer would often like the music to be more frantic. This general concept of an "intensity level" is often difficult to communicate to the programmer and even more challenging to implement convincingly in an application.
Variable Intensity Music and Ambience
DirectMusic provides several potential solutions for creating dynamic music. In this chapter, we cover two in particular — the use of DirectMusic Style files and the use of Segment-based branching.
DirectMusic Style Files
Perhaps the easiest way to implement intensity level is to use DirectMusic Style (.sty) files, which contain numerous musical fragments, each with an assigned intensity level. Audio producers compose one or more patterns (musical loops) within a Style. A pattern is composed of a Pattern Track, those same MIDI-supporting Tracks we've already used in Segments. Each pattern is assigned a groove level range, DirectMusic's term for musical intensity. When the programmer alters the groove level based on application events, a new pattern is chosen automatically based on the new intensity requested.
When a new Style file is created (File>New…, or the Ctrl+N shortcut), it appears in the project tree with several folders.
Figure 4-1: A newly created style file. Remember that design-time file extensions end with p, so this is a .stp file. The run-time version of the file will be a .sty file.
The Bands folder contains one or more Bands (information about per-channel volume, pan, and patch change settings) that can be dragged into a Segment's Band Track. Typically, you will just use one Band per Style, and that Band is automatically inserted when you drop the Style into a Segment. More information on that in a bit.
Motifs provide a subset of the functionality available to secondary Segments (and for history buffs out there, they are actually the predecessor to secondary Segments). Recall that one or more secondary Segments can be played at a time and are typically layered on top of a primary Segment, whereas only a single primary Segment can be playing at a time. While motifs may appear somewhat convenient (all living in a single file), there are several significant drawbacks to motifs:
They cannot be accessed directly using DirectX Audio Scripting. DirectX Audio scripts can only play and stop Segments, not motifs (which reside in Styles).
While they can be played directly by a programmer, the composer needs to manually embed a Band within the motif to ensure that its expected instrument patch changes are applied before it is played.
While motifs support variations, it's up to the game programmer to keep using the same instance of the motif in order to avoid resetting the variation order.
In short, for most cases, secondary Segments are much more useful and convenient than motifs.
So we're typically going to have a Style with a single Band, no motifs, and several patterns. Our default Style has a single pattern. If we double-click it, the Pattern Editor (looking quite similar to the Segment Designer window) opens. MIDI controller editing is identical to the behavior in Pattern Tracks in Segments.
Figure 4-2: The Pattern Editor window.
There is a new Track at the top of the part called the Chords for Composition Track. This Track, covered in more detail in Chapter 6, tells DirectMusic what the original function of the music was so that DirectMusic can make an educated choice when revoicing the music to work over a new chord or key. For now, we leave it at the C major default.
Note DirectMusic Style files only support Pattern Tracks and the Chords for Composition Track and therefore DLS Instruments. In particular, Styles cannot be used in conjunction with Wave Tracks, so consider Segment-based branching if you are using prerendered and/or streamed music.
Opening the properties page for a pattern gives us access to groove range and other pattern-related functions. This is also where you can adjust the length of this specific pattern.
Figure 4-3: The properties page for a pattern within a Style.
We've discussed groove range earlier. An audio producer can assign a pattern to a single groove level if they wish or a range of appropriate groove levels. Typically, a range of values is used. This allows for two things:
Pattern variability. If two patterns have overlapping ranges, a requested groove level within the overlap will randomly choose between the two patterns (all other aspects of the patterns being equal — see below for more details on how a pattern is chosen).
Easy addition of patterns later. Let's say the composer creates three patterns — a low-intensity pattern (assigned to groove range 0–25), a medium-intensity pattern (assigned to groove range 26–75), and a high-intensity pattern (assigned to groove range 76–100). After trying the application, the composer might decide another pattern with intensity between "medium" and "high" is appropriate. He can create a new pattern without the programmer needing to make any changes to his code. By contrast, using a single groove level, rather than a range, for each pattern would probably require the programmer to change how he determines the groove level.
The Destination Range option can only be used when you know what groove level your music is heading toward — that is, generally only when groove levels are authored directly into a Segment via a Groove Track (see the "Using Styles with Segments" section later in this chapter). With destination range, DirectMusic will actually "look ahead" to what groove level is going to be needed next before choosing which pattern should be played. While this could be somewhat useful when "ramping" from one groove level to another, the idea of authoring groove levels into content is typically contrary to the desire for dynamic manipulation of groove levels. For this reason, destination range is rarely used.
The Embellishment area allows you to add even further control to how a pattern is chosen. Embellishments are generally specific patterns that you can call for without changing groove levels. For instance, you might be playing a piece of music at groove level 50 and you want to transition to a new piece of music, also at groove level 50. Rather than temporarily changing the groove level to pick a convincing ending pattern, you could create a pattern still assigned to groove level 50 that is an end embellishment. To further strengthen the flow of the transition, the new Style could have an intro embellishment pattern. Then when the programmer or scripter wants to start playing the second piece of music, he can say to play ending and introductory embellishments, and these specially assigned patterns will be played between the two pieces of music. The naming conventions for embellishments (intro, end, fill, break, and custom) are actually somewhat arbitrary — from a programmer's or scripter's point of view, there's no difference between most of them. The exceptions are that an intro embellishment is always picked from the Style you are going to, and the end embellishment is always picked from the Style that you are coming from. Of course, if you're sticking with the same Style and just wanted a brief pattern change, even these have no difference in function. Note that custom embellishment types (with a range from 100–199 to help keep them from becoming confused with groove ranges) can only be requested programmatically or within content; DirectX Audio Scripting does not support them.
The Chord Rhythm option adds yet another layer of control over how DirectMusic chooses which pattern to play, assuming the Segment uses chords (in Chord Tracks or via ChordMaps). The audio producer might have one pattern that is more ideal to a rapidly changing progression and another that works better under a static chord. DirectMusic will look for a pattern with a chord change rhythm that maps closest to the currently playing Segment's chord rhythm.
Choosing a Pattern from a Style
So after all of these settings, how exactly does DirectMusic choose which pattern to play at a given time? In descending order of priority:
Look for the embellishment requested.
Look for the groove level requested.
Look for pattern matching destination groove level to next groove change. Again, as destination groove level is rarely known ahead of time, this particular rule is rarely used.
Longest pattern that fits available space (between now and end of Segment or next groove change).
Look for a pattern with an appropriate chord rhythm.
In case of a match in all other settings, randomly choose one of these matching patterns to play.
The fourth rule is often the most significant to keep in mind. The argument here is that DirectMusic should pick the musically "strongest" progression, which would mean to prefer to play a single eight-bar pattern rather than a one-bar pattern eight times. But the rule also applies when you have multiple longer patterns to choose between. If you had been anticipating using variable length patterns, it's generally not easy to get around this restriction but it does allow for a simpler composition and musical transition process.
Building Up Styles
The most basic use for Styles is what we like to call additive instrumentation. Each pattern is authored identically to the previous (lower groove level) one but adds another instrument to the mix. This can work quite effectively and is relatively easy to implement. The audio producer creates a low groove level pattern (which could be silence, basic rhythm, or perhaps just subtle ambience). They then create additional patterns, each at a higher groove level and each adding a new instrumental line or additional rhythmic complexity.
Primarily due to the pattern-choosing rules discussed above, it is often easiest to create every basic pattern with the same length. Embellishment patterns of differing lengths can help break up the rhythm somewhat, though all patterns must use the Style's global time signature for their playback length. Individual parts in a pattern can be varying lengths and time signatures to help distract from the feeling of constant time signature.
By the way, groove level and pattern length can be viewed in a somewhat easier-to-read interface via the Style Designer window. The window is opened by double-clicking on the Style itself in the project tree rather than any of its patterns and displays information on all of the patterns (their length, groove ranges, embellishments they are assigned to, and so on).
Figure 4-4: The Style Designer window for a Style with five patterns for varying groove levels. The highlighted pattern is an ending embellishment (as indicated by the E in the Embellishment column).
Using Styles with Segments
Remember that the basic unit of playback (as far as the programmer is concerned) is typically the DirectMusic Segment, not a Style file. So we're going to want to create a Segment that actually "plays" our style. Several Track types come into play when instructing a Segment on how to use one or more Styles:
Style Track: This Track is the basis for using Styles with Segments. Typically, the Style Track will consist of a single Style placed in the first bar. However, there's nothing wrong with changing Styles as a Segment proceeds.
Groove Track: This is the Track where the audio producer can set the current groove level for the Segment. This Track also allows for a Segment to manually request an embellishment pattern (for instance, play an intro embellishment at the beginning of the Segment). Inserting a Style into a Style Track automatically makes changes to several other Tracks in a Segment, inserting new Tracks if necessary.
Time Signature Track: This Track becomes locked to the time signature of the current Style, as you can't override a Style's time signature in a Segment. If additional Styles are inserted later in the Segment, similarly locked time signature events (which appear "grayed out") will appear.
Tempo Track: The default tempo for the Style (specified in the Style's property page and the Style Designer window) is inserted on the first beat of the bar where the Style was entered.
Band Track: The Style's default Band is inserted one tick before the bar where the Style begins playback. This ensures that all of the instrument changes and volume/pan settings are applied just before the first notes are heard (assuming no notes begin before beat one — if any notes do occupy the pickup bar, you should manually move the Band even earlier). Note that the Band inserted into the Band Track is a copy of the Style's Band, not a "reference" to it. That is, if you now go back and edit the Style's Band, those changes are not applied to the Band in the Segment. Similarly, if you edit the Band in the Segment, those changes won't be reflected in the original Style's Band. We typically manually drag the Style's Band into the Segment just to make sure we are not using an old version of the Band.
So all of these Tracks are automatically filled in for us. But what about the Groove Track? Because the groove level is something we want to be dynamically changed by the programmer while the application is running, composers don't typically author groove changes into the Groove Track beyond an initial setting (typically the lowest groove level) in the first bar.
Figure 4-5: A typical Style-based Segment. Groove level 1 is specified in the first bar. When the Style was inserted, the other tracks were created and/or modified automatically.
Before we completely dismiss the Groove Track, it is still quite useful for auditioning groove changes within DirectMusic Producer. After composing a bunch of patterns of varying groove levels, you might create a long Segment, drop in the Style and a few groove changes, and listen to how the performance sounds. You could even use variable groove changes (the +/– option in a groove change's property page) or experiment with the embellishment types.
As far as the length of the Segment, you have several options. If you are using patterns of varying lengths, you'll generally want to make the Segment the least common multiple of those lengths. For instance, if one pattern is four bars and another is three, you'll want to make the Segment 12 bars long (and infinitely looping). This just makes sure that, assuming a single groove level (and often a single pattern) is played for a long time, the pattern won't be interrupted or cut short by several bars due to the Segment ending. Worse, remember that the pattern chosen is also influenced by how much space is left before the end of the Segment — so you might get a completely different pattern chosen right at the end if the pattern length did not match the Segment length.
For the specialized (and easier to manage) case where all patterns have exactly the same length, creating an infinitely looping Segment of that same length is generally preferable. This will also give us some added flexibility in how transitions between the groove levels occur, particularly with the ability to successfully use alignment transitions.
Changing the Playing Patterns
If you've experimented with groove changes to audition your patterns in a Segment, you might notice that the new groove level's pattern didn't always take over immediately. The reason behind this is that a groove change tells DirectMusic to choose a more appropriate pattern when the currently playing pattern finishes. Certainly, the application might change groove levels quite a bit, and you wouldn't want new patterns abruptly starting and restarting frequently. However, if you've created a lot of 20-bar patterns, the change to the new groove level could definitely take quite a while and more time than you would hope.
There are several options for making pattern changes occur more frequently. The easiest will involve the programmer or scripter. Every time he adjusts the groove level, he can also restart the currently playing Segment using any of the transition types we've already discussed. For instance, the audio producer could instruct the programmer to restart the Segment at the next bar. This means that a groove change will "take effect" (a new pattern will begin playing) within the next bar. Or, in the above example, the audio producer could structure his 20-bar patterns such that they're actually composed of five four-bar-long sections. The audio producer could then put a Marker Track in the Segment (note that the markers are not in the Style file, as Style patterns don't support Marker Tracks) at the natural break points for the music, and the programmer could restart the Segment on a marker boundary. One difficulty with this solution is that we are restarting the Segment (and thus whatever the appropriate pattern is) every time we touch the groove level. This can be problematic when smaller groove level changes shouldn't really affect the pattern we're playing. For instance, if our "low level" pattern spans groove levels one through 25, it doesn't really matter to us if the groove level changes from one to two. The same piece of music should just keep playing. But in the simplest implementation of the above solution, we're retriggering the Segment — and thus restarting the pattern — every time the groove level changes. Depending on how frequently these changes occur, we could get very intimately acquainted with the first few bars of the pattern and never hear the 20th onward. One way to combat this is for the content creator to instruct the programmer as to when the Segment should really be restarted versus allowing it to continue with the newly adjusted groove level.
Based on the above issue with potentially restarting the playing pattern from the beginning constantly, another commonly used technique is to use alignment to Segment transitions. Remember, in this kind of transition, we jump to the same relative location in the destination Segment as we were at in the previous Segment. So if we were at bar four beat two, we jump to bar four beat two of the new piece of music. In the particular case of adjusting groove levels, both the source and destination Segment are the same piece of music. The retriggering is enough to bump us over to a new pattern more appropriate for the current groove level. But with the use of alignment, we transition over in mid-pattern rather than restarting the Segment. This technique is a bit trickier to manage than the previous one. Remember that if we're starting in the middle of a pattern, any notes that would have started playing prior to that point would not be retriggered — so for instance, if our destination had some nice string pads that had started on the downbeat, we wouldn't hear them if our transition occurred on beat two. An added complexity occurs if our patterns had different lengths, as they now may transition to musically awkward positions in each other. This is another reason why it's often easiest to keep all patterns at the same length.
Figure 4-6: Attempting to transition using Segment alignment can be difficult for patterns of differing lengths. In the above diagram, Pattern 1 is four bars long, and Pattern 2 is five bars long. Below them is a display of the 20 bars of a looping Segment, so we can see how they line up over time. If we transition from Pattern 1 to Pattern 2 at, say, the end of bar eight, we're transitioning at the end of Pattern 1 but into the start of the fourth bar of Pattern 2. If both patterns had been the same length, we would be making a potentially more natural transition from the end of Pattern 1 to the beginning of Pattern 2.
Segment-Based Branching and Layering
An alternative to DirectMusic Style files for creating variable intensity music and ambience is to use multiple Segments. Remember that with strictly Style-based playback, you are restricted to DLS instruments. With Segments, you have the ability to mix and match between DLS instruments and prerendered streaming wave files. There are two primary ways that Segments can be used to shift intensity: branching and layering.
Branching Playback
In the case of branching Segments, we change the Segment we are playing at a musically appropriate boundary, based on some event that changes the emotional level we want to convey. In the simplest implementation, branching just consists of authoring complete pieces of linear music for each emotional intensity, and then picking logical places where one piece could jump to the beginning of the next. Often measures are appropriate boundaries; if not every measure is, the composer can use markers to dictate which are.
Sometimes the above can result in abrupt and unsatisfying transitions, particularly if the two pieces of music were not written with each other in mind. The jolt from playing a slow-moving piece of music to a pulse-poundingly fast one might be appropriate in some circumstances, but in others it can distract more than intended. A potential solution here is to use transition Segments.Aswepreviously discussed, when you tell a new Segment to start playing, you can specify a transition Segment that should be played before this new Segment starts. A common solution to the abruptness issue is to create quick little "bridges" that ramp up or down the tempo, smooth out any instrument entrances and exits, and so on. Of course, this can lead to a bit of extra work on the composer's part — you might find yourself writing transition Segments for every possible configuration (slow to fast, fast to slow, slow piece 1 to slow piece 2, slow piece 2 to slow piece 1, and so on) unless you're clear as to what pieces could possibly jump to other pieces. Building up a roadmap beforehand can aid both in the planning and authoring of transition Segments.
Figure 4-7: A sample diagram for five pieces of music. Each arrow indicates that a piece of music will need to be able to transition to another piece of music. For this scenario, we assume that you have to somehow "engage" enemies in combat, so you'll never jump right to the "fighting" music from the "no enemies" music. Similarly, if you decide to flee the room, you'll disengage from the enemies first, so you don't have to worry about the "fighting" music jumping directly to the hallway "ambient" music.
Yet another alternative use of branching is Segment alignment. Remember that by default, a new Segment will start playing from the beginning. Using alignment, we can make it start at the same position where the last Segment left off. With this technique, we can achieve results similar to the Style-based playback above, but with the benefit of streamed waves. Using this technique, you could author several different versions of the same audio track, with different mixes and increased instrumentation in each version. Again, using markers (in marker tracks) to control what are appropriate boundaries, the music can seamlessly ramp up or down to another wave based on the scenario.
Layered Segments
The other effective Segment-based tool for creating dynamic music is layering. Here we keep playing the same basic piece of music, but we layer other pieces of music over the top. This involves the use of secondary Segments. Remember that only one primary Segment can be playing at a time, and playing a new primary Segment automatically stops any currently playing ones. But you can layer as many secondary Segments on top of a primary Segment as you like. Don't forget that whether a Segment is played as primary or secondary is in the hands of the programmer or scripter, so the composer needs to communicate this information beforehand.
A common use of layered secondary Segments is as motific elements. Indeed, these secondary Segments replace the somewhat outdated Style-based motifs, which were much less flexible and required extra effort from the programmer or scripter in order to use them. As an example, let's say our character's love interest enters the room, and the love theme plays on top of the basic music track. The use of Chord Tracks (described in more detail in Chapter 6) can make this all the more effective. The love theme can be played in the appropriate key, and even repitch itself based on the chord progression of the underlying music.
As a brief example, let's take any old Segment we've already authored and add a Chord Track to it if it doesn't already have one. (Right-click on Segment > Add Track(s)…, and choose Chord Track from the list.) Now let's right-click on the Chord Track (the leftmost part of the track that says "Chords") and bring up its properties. Odds are the track thinks our piece of music is in the key of C because we haven't told it otherwise. Mine happens to be in Eb, so I update the property page like so:
Chord Track Properties dialog.
We'd probably also want to adjust that C major 7 chord that appears in measure 1 by default. Let's make it an Eb major 7 instead. (I'm not going to bother entering the full chord progression, but we'll use this particular chord in a moment.) We right-click on the CM7, choose Properties, and modify the property page like so:
Our Eb major seventh chord. Adjusting layer 1 (the bottommost chord layer) defaults to automatically editing all four layers, so you would only need to adjust the chord (four-octave keyboard on the left) and the scale (one-octave keyboard on the right) once. Right-click and uncheck Auto-Sync Level 1 to All to change this behavior.
Now let's create our romantic theme. We create a new Segment with a Pattern Track and a Band Track (making sure that the Band and pattern pchannels don't interfere with any from our original piece of music). As far as the actual music, I'll just borrow a recognizable theme from Tchaikovsky's Romeo and Juliet:
Our (well, Tchaikovsky's) love theme. Note that we've authored it in the key of C (not having a Chord Track implies this) and named the Segment "LoveTheme." The Pattern Track part uses performance channel 17 (which can be adjusted from the property page), and we disabled all variations except our theme on variation 1. Our band similarly just has an entry for performance channel 17 telling it what instrument to play.
If we played our theme as a primary Segment, it would play in C major. But if we play it as a secondary Segment against our piece of music in the key of Eb, it will transpose to this key. To try it out, open up the Secondary Segment Toolbar (from the View menu, choose Secondary Segment Toolbar), and choose our Segment from one of the drop-downs.
The Secondary Segment toolbar.
You can specify the boundary it will audition on by right-clicking the Play button. Now play the primary Segment from the main transport controls (click on the Segment to give it "focus," which will make its name appear next to the primary segment's Play button). As it plays, play our love theme from the secondary Segment toolbar's Play button. The secondary Segment transposes to fit the key of our primary Segment. Similarly, this theme could be played against any of our linear compositions in the proper key just by authoring a Chord Track into those Segments and specifying the proper key and scale in a chord in the track.
Transposition is one thing, but as a more advanced use, we might want our theme to actually follow the chord progression of our primary Segment. Let's add another chord to our primary Segment and start off with a dominant-tonic (V-I) progression. We'll move that Eb chord to bar two (by dragging or copying and pasting) and insert a Bb chord in bar 1.
Now when we play our secondary Segment as this primary Segment plays, the notes will conform to the current chord as well as the scale. We can adjust the specifics of how our secondary Segment adjusts itself from its property page, and can even override these settings on a per-note basis if we wish.
The Default Play Mode area determines how our secondary Segment responds to chord changes. In the default behavior, chord and scale are used to determine appropriate notes, but there are also settings to only transpose according to scale, to remain fixed to the authored notes (useful for drum tracks), and so on.
Alternatively, secondary Segments can be played as controlling secondary Segments. In this case, rather than taking on the characteristics of the primary Segment, they force the primary Segment to take on their characteristics of tempo, chord progression, and so on — whichever tracks are present in the secondary Segment are used to replace to the primary Segment's. Going back to our previous example, we can add a Chord Track to our secondary Segment telling DirectMusic that our theme was indeed authored in the key of C, overaCmajor chord. If we then played it as a controlling secondary Segment over the primary Segment, the primary Segment would now take on the chord progression of the secondary Segment.
Adding a new Chord Track and C major chord to our secondary Segment.
Setting up a Play button in the secondary Segment toolbar (by right-clicking on it) to play a controlling secondary Segment. Don't forget that this is an audition setting only; the composer would want to tell the programmer or scripter that the Segment should be played as a controlling secondary Segment.
Remember that Sequence Tracks do not respond to chord changes — they are static, non-pitch-varying tracks, analogous to traditional sequenced MIDI tracks. So the fact that our controlling secondary Segment has switched out the chord progression won't be heard in the primary Segment unless any of its notes are played by Pattern Tracks. But if our music was placed in a Pattern Track, and we had traced out the chord progression for the music in our Chord Track, the primary Segment would sound quite different over this new chord progression. To repeat a useful scenario for this, background music authored in a major key could switch to a minor key when the villain enters the room, just by playing a controlling secondary Segment composed of minor chords. Indeed, there is nothing that says a secondary Segment has to be composed of any notes at all — it is perfectly acceptable to use them for Tempo, Chord, Style, Wave, and any other tracks.
In this chapter, we have attempted to supply an overview and a basic understanding of some of the power and flexibility DirectMusic can bring to an interactive application. It can be used simply to provide a MIDI-capable synthesizer, with the ability to deliver instrument collections that ensure uniform content playback across multiple sound cards. It can be used to synchronously play back real-time-rendered (MIDI) and prerendered (wave) sounds in synch with each other. DirectMusic provides the ability to seamlessly transition from one piece of music to another at appropriate (and content-defined) boundaries. And it gives a rich set of tools to composers and sound designers for creating variation and dynamically adjustable content. Being aware of some of the rules and restrictions for how it can best be used, I encourage you to experiment with the concepts presented here. It's often easiest to gain familiarity with a few tools at a time rather than being overwhelmed by the options; this chapter has merely touched the surface of the features DirectMusic supports.
Linear Playback
Overview
Scott Selfon
Okay, let's get cracking. The first thing we want to do is be able to play sound. We add variability, dynamism, and the rest later. Let's focus on playing back a wave file.
Note At this point, if you have not already, you must install DirectX 9.0 as well as DirectMusic Producer, which are available on the companion CD.
Waves in Wave Tracks
To play back a wave file in DirectMusic, you'll create a Segment out of it. As already mentioned, the DirectMusic Segment file (.sgt) is the basic unit of DirectMusic production. Segments are built from one or more of DirectMusic's various track types, which can make sound (via stand-alone waves and/or DLS instrument triggering) or modify performances (tempo, chord progression, intensity level, etc.). You can create a Segment out of a wave file in DirectMusic Producer. Run DirectMusic Producer and go to File>Import File into Project>Wave File as Segment…. Open a wave file into the program. Building a Segment from a wave file, we get our first look at one of the basic track types in DirectMusic — the Wave Track. Beyond the 32 variation buttons (which we cover in Chapter 3), Wave Tracks play along the sequencer timeline with other Wave Tracks and also with MIDI sequences.
If you look at the size of the Segment file, it is much smaller than the wave file. Here is an important early lesson in content delivery: Segments will know where the wave and sample files that it needs to play are but do not store those files as part of itself. There are several reasons to do this; if several Segments use the same wave data, you do not have to worry about having two copies of it in memory. In addition, if you later want to go back and edit that wave data, you do not have to worry about copying it into several different places. That said, there are various reasons that you might instead want to embed the wave data within the Segment itself — for instance, for the convenience of only having to deliver a single file or file load time considerations.
Files with the extension ".**p" are design-time files. Design-time files are used in DirectMusic Producer for editing purposes and contain information not necessary for run-time use — for instance, a Segment used in a game does not need to include information on what size and position to open editing windows. In addition, design-time files always reference the content they use, even if you specify that content should be embedded. For these reasons, when content is meant to be integrated into a game or a special player, you should save the Segment as a run-time file (either via the per-file right-click menu Runtime Save As… option or the global Runtime Save All Files option from the File menu). When Segments are run-time saved, you will see the .sgt extension, and wave files will similarly have the more expected .wav extension. So to summarize, be sure to save your Segments as design-time (.sgp) files while you are working on them and as run-time files (.sgt) when they are finished and ready for distribution. For more details on content delivery, there is a white paper available on the Microsoft Developer Network web site (msdn.microsoft.com) called "Delivering the Goods: Microsoft DirectMusic File Management Tips."
Streaming Versus In-Memory Playback
A minute-long 44.1 kHz 16-bit stereo wave file is 10MB. 10MB isn't a big deal when you run a stand-alone DirectMusic file on a contemporary computer, but it is a very large file in the world of game audio. Do not forget that the rest of a game's resources need to reside in memory as well. If the game is written to run on a system with as little as 64MB of RAM, you are already in way over your head. Console games are even more unforgiving. Consider yourself lucky if you get 4MB for your entire sound budget! You can use audio streaming to alleviate these restrictions. Streaming is a technique that works a lot like a cassette player. In a cassette player, audio data is moved across the play head a little bit at a time. The play head reads the data as it comes and plays the appropriate sound. With streaming, a small area in memory called a buffer is created. Wave files are moved, bit by bit, through the buffer and read by DirectMusic. The CD player in your PC uses streaming for playback. If it weren't for streaming, you'd have to load the entire music file into your computer's RAM, which in most cases simply isn't an option.
DirectMusic uses the following rule for streaming: The wave streams with 500 msec readahead if it is more than five seconds long. If it is shorter than five seconds, it resides and plays from memory (RAM). Readahead determines the size of the buffer. For 500 msec, our 44.1 kHz 16-bit stereo wave file will use 88.2KB of memory (.5 sec x 44100 samples/sec x 2 bytes/sample x 2 channels), a big difference when compared to 10MB! Memory usage is reduced by a factor of more than 100! You can override this behavior, choosing to load all wave data to memory, or specify a different readahead value in the Compression/Streaming tab of a wave's property page in DirectMusic Producer. To get to any Track's property page, simply right-click on it in the Track window.
Tracks and Parts and Layers, Oh My
Before we move any further, the concept of Tracks and parts can use a bit of sorting out. When we created our Segment from a wave file, it consisted of a single Track, a Wave Track, with a single part (by default on pchannel 1). Each part specifies its own variation behavior, what pchannel it is played on, and, in the case of Wave Tracks, volume for that part as well. By comparison, Tracks specify big picture (or "global") behaviors such as clock-time, as well as more exotic settings like multiple, concurrent Track groups. If you open the property page for a Track or part, you'll notice the separate tabs with their own settings — the Track properties actually includes both the Track Group Assignments and Flags tabs in addition to the actual Track properties tab.
Remember that waves in Wave Track parts never conflict with DLS Collections on a pchannel; you can play as many waves as you want simultaneously and still play a DLS Collection instrument on the same pchannel without any problem. The reason a Wave Track part can be assigned to a pchannel is that MIDI controllers can still be used to drive wave playback (for instance, pan on a mono wave, pitch bend, volume, etc.). The pchannel that a Wave Track is assigned to can be altered in the wave part's property page.
This brings us to the concept of layers, or the various lettered rows (a, b, c, etc.) that you see displayed in DirectMusic Producer for a single Wave Track part. As mentioned, you can play as many waves at one time as you wish. Therefore, layers are purely an aid for placing waves against the Segment timeline, rather than having many waves overlap on a tiny area of the screen and not be able to easily edit them (or tell which wave started and finished when). Waves can be placed on different layers within a part for easier legibility. All layers are equal, are played simultaneously, and do not take any additional resources (processor power or memory) when played.
Figure 2-1: Multiple waves on Wave Tracks are split into "layers" so you can see their beginnings and endings more easily.
As a last bit of terminology for now, certain parts are actually subdivided even further into strips. In particular, the parts for MIDI-supporting Tracks (Pattern Tracks and Sequence Tracks) have separate strips for note information and continuous controller information. Pattern Tracks also have an additional variation switch point strip, which we cover in Chapter 3 when we discuss variations.
MIDI and DLS Collections
Creating a DirectMusic Segment from a piece of MIDI music is simple. In DirectMusic Producer, follow File>Import File into Project>MIDI File as Segment…. This creates a DirectMusic Segment from a MIDI file, importing note, MIDI controller, and other pertinent data along the way. Let's examine some new Track types related to MIDI:
Tempo Track: Specifies the current tempo for the piece of music. You can override this by playing a primary or controlling secondary Segment with its own tempo settings.
Time Signature Track: Sets the time signature for the piece. Use this to track where measures fall as well as how many subdivisions (grids) to give each beat.
Chord Track: Specifies the key as well as specific chord progressions for a piece of music. Typically, for an imported MIDI file, this will just indicateaCmajor chord. DirectMusic Producer does not attempt to analyze the imported MIDI file for chordal information.
Sequence Track: A sequence is what its name implies. Sequence Tracks house MIDI sequences. This is where the majority of MIDI information is imported. Notice that as with Wave Tracks, Sequence Tracks can (and typically do) consist of multiple parts. By default, each MIDI channel is brought in as a separate part on the corresponding pchannel. In addition, each part can contain its own continuous controller information. Unlike Pattern Tracks (more on these in Chapter 3), Sequence Tracks are linear, non-variable sequences; they play the same every time, and they do not respond to chord changes (though they do respond to tempo changes).
Figure 2-2: A Segment with a Sequence Track consisting of multiple instruments.
Band Track: Bands are how Performance channels refer to DLS instrument collections. The Band Track is an area where initial pan, volume, and patch change settings for Tracks are stored. This is often an area of confusion, as you can also have volume and pan controller data within Sequence Track (or Pattern Track) parts. The Band Track typically just has a single Band with the initial settings. Subsequent volume changes are typically created as continuous MIDI controller events in Sequence or Pattern tracks. Continuous controller events, unlike band settings, allow you to easily sweep values over time (for example, to fade a track in or out). If any patch changes occur during performance, another Band can be placed in the Band Track, though the elimination of MIDI channel limitations means there is often little reason not to just place each instrument on its own unique and unchanging channel.
Building DLS Collections
Our Segment is now ready for playback. Of course, we're assuming that the piece of music will be played using a preauthored DLS Collection, such as the gm.dls collection that comes with Windows machines. Otherwise, it will be trying to play instruments that don't exist on the end user's machine, and these instruments would be played silently. If we wanted to use our own instruments, we would want to build one or more DLS Collections. DirectMusic Producer provides this authoring ability in DLS Designer. Alternatively, the DLS-2 format is a widespread standard, and there are several tools out there for converting from other common wavetable synthesizer formats to the DLS format.
Creating DLS Collections is often one of the more challenging tasks when you decide to create real-time rendered (versus streamed prerendered) audio. Remember that unlike streamed audio, your DLS Collection will occupy system memory (RAM), which is typically one of the most precious commodities for an application. For this reason, you'll want to create collections that get you the most bang for your buck in terms of memory versus instrument range and quality.
Let's create our first DLS Collection. From the File menu, select New, and from the dialog that appears, choose DLS Collection and hit OK. We're presented with a project tree entry for our DLS Collection (using the design-time .dlp file extension), which has two subfolders, Instruments and Waves.
Figure 2-3: An empty DLS Collection in the project tree.
DLS Collections are composed of waves (effectively, ordinary wave files) and instruments, which consist of instructions for how waves should map to MIDI notes, along with a fairly sophisticated set of audio-processing features. Let's add a few waves for our first instrument, a piano. We can drag our waves right into the Waves folder from a Windows Explorer window or right-click on the Waves folder and choose Insert Wave… from the menu that appears.
Figure 2-4: We've inserted four wave files into our DLS Collection.
We can now adjust the properties for these waves by right-clicking on them and choosing Properties to bring up their property window. The most important settings to note are going to be Root Note, loop points (both in the Wave tab), and compression settings (in the Compression/Streaming tab). We'll return to compression a bit later. Root Note specifies the base note that this wave corresponds to, also known in other wave authoring programs as the "unity pitch" or "source note." For our above piano sounds, determining the root note was made easier by including the root note information right in the wave file names. DirectMusic Producer will also automatically use any root note information stored with the wave file, which some wave authoring tools will provide. Otherwise, we can adjust the root note manually from the property page, using either the scroll arrows or by playing the appropriate note on a MIDI keyboard.
Figure 2-5: Setting the proper root note for our BritePiano_C4 wave.
This particular piano wave doesn't loop, so we don't need to worry about the loop settings. Again, if loop settings had been set on the source wave file, DirectMusic Producer would automatically use that information to set loop points.
DirectMusic Producer provides some basic editing features on waves in the Wave Editor, which can be opened by double-clicking any wave in the Waves folder (or indeed, any separate wave file you've inserted into your DirectMusic project).
Figure 2-6: The Wave Editor. You can specify whether waveform selections (made by clicking and dragging) should snap to zero crossings via Snap To Zero, and you can specify a loop point by selecting Set Loop From Selection.
The Wave Editor window supports clipboard functions, so you can copy to and paste from other wave editing applications if you wish. You can also try out any loop points you've set (or just hear the wave you've created) by playing the wave. As with Segment playback, you can use the Play button from the Transport Controls toolbar, or use the Spacebar as a shortcut to audition your wave. Note that you can only start playing your waves from the beginning, regardless of where you position the play cursor, as DLS waves cannot be started from an offset.
Now that we've set up our waves, let's add them to a DLS instrument so we can play them via MIDI notes. To create an instrument, right-click on the Instruments folder and choose Insert Instrument.
Figure 2-7: Creating an instrument.
You'll notice that each instrument is assigned its own instrument ID, a unique combination of three values (MSB, LSB, and Patch) that allows for more than 2 million instruments — a bit more freeing than the traditional 128 MIDI patch changes. DirectMusic Producer will make sure that all loaded DLS Collection instruments use unique instrument IDs, but you should take care if you author your DLS Collections in different projects to make sure that they don't conflict. Otherwise, if two instruments with identical IDs are loaded at the same time, DirectMusic will have no way of knowing which one you want to use. The General MIDI DLS Collection (gm.dls) that is found on all Windows machines uses 0,0,0 through 0,0,128, so our new instrument probably defaulted to instrument ID (0,1,0).
Let's open up the Instrument editor by double-clicking on the instrument in the project tree.
Figure 2-8: The Instrument editor.
There are lots of options here that really demonstrate the power of a DLS-2 synthesizer, but for now let's just start with our basic instrument. The area above the piano keyboard graphic is where we define which wave or waves should be played for a given note. Each wave assigned to a range of notes is called a DLS region. As you can see, our new instrument defaults to a single region, which plays the wave BritePiano_C5, over the entire range of MIDI notes. The piano keyboard note with a black dot on it (C5) indicates the root note for the region. Remember that the root note will play the wave as it was authored, notes above the root note will pitch the wave up, and notes below the root note will pitch the wave down.
Looking along the left edge of our region, you can see the labels DLS1, 1, 2, and so on. These labels identify the various layers of this instrument. Layers allow you to create overlapping regions, one of the nice features of the DLS-2 specification. One of the more common uses is for adding multiple velocity versions of a wave, where as the note is struck harder, the instrument will switch to a different wave file. This is particularly effective for percussion instruments. Notice that each region has a velocity range that can be set in the Region frame. Multiple layers also mean that a single MIDI note could trigger a much more complex instrument composed of several simultaneously played wave files. Remember that regions on a single layer cannot overlap. The DLS1 layer is the single layer that was supported in the original DLS-1 specification, and is generally still used to author any single-layer DLS instruments. For this simple instrument, we won't worry about using multiple layers for the moment.
Our first step is to create regions for the rest of our piano waves. We want to resize that existing region so there is room on the layer for the other waves (that, and we probably don't want to try to play this wave over the entire range of the MIDI keyboard!). If you move the mouse over either edge of the region, you'll see that it turns into a "resize" icon, and you can then drag in the outer boundaries of the region.
Figure 2-9: Resizing a region with the mouse. Alternatively, we could change the Note Range input boxes in the Region frame.
By the way, this is similar to the functionality for note resizing (changing start time and duration) in the MIDI-style Tracks (Pattern Tracks and Sequence Tracks). There are other similarities between region editing and note editing — the selected region turns red, just as notes do. The layer we are currently working with turns yellow, just as the current Track does in the Segment Designer window. And just as you can insert notes by holding the Ctrl key while clicking, you can create a new region in the same way.
You can resize the region as long as the mouse is held down and dragged (or, as before, resize the region by grabbing a side edge of it). As an alternative to drawing a region with the mouse, you can right-click and choose Insert Region from the menu that appears.
Every region that we create defaults to use the same wave that the first region did. So we'll want to choose a more appropriate wave from the drop-down aptly labeled Wave. Let's assign BritePiano_E5 to this new region we've created.
We repeat the process for the rest of our waves to build up our instrument. Notice as you click on each region that you can view its particular properties for wave, range, root note, etc., in the Region frame.
Figure 2-10: The completed regions for our piano instrument.
There are several potential issues to discuss here. First, should all instruments span the entire keyboard? On the plus side, the instrument would make a sound no matter what note was played. On the minus side, when the source wave is extremely repitched (as BritePiano_C4 is in the downward direction and BritePiano_E5 is in the upward direction), it can become almost unrecognizable, dramatically changing timbre and quality. Most composers opt for the latter, if only to know whether their music ever goes significantly out of an acceptable range for their instruments. If it does, the music can be adjusted (or the DLS instrument expanded to include additional regions).
A second question is how far above and/or below the root note a region should span. That is, does the wave maintain the quality of the source instrument more as it is pitched up or down? This can vary significantly from wave to wave (and the aesthetic opinions of composers also differ quite a bit). For the above example, we did a bit of both — the bottom three regions encircle notes both above and below the root note, while the highest piano region (our BritePiano_ E5 wave) extends upward from the root note. How far you can "stretch" a note can vary quite significantly based on the kind of instrument and the source waves.
Once you've created your instrument, you can try auditioning it in several ways. An attached MIDI keyboard will allow you to trigger notes (and indeed, small red dots will pop up on the keyboard graphic as notes are played). You can also click on the "notes" of the keyboard graphic to trigger the note. The Audition Options menu allows you to set the velocity of this mouse click and choose whether to audition all layers or only the selected one. (In the case of our example, we only have one layer, so the two options don't matter.)
Try out the piano instrument, paying particular attention to the transitions between regions. These are often difficult to match smoothly, as the pitch shifting of the waves begins to alter their characteristics and the instruments can begin to develop musical "breaks" where the tone quality changes significantly. If you are writing a purely tonal piece of music, it is often useful to author your DLS Collections such that the tonic and dominant tones (the scale root and the 5th degree) are the root notes of your regions. That way, the "best" version of each wave (not repitched) will be played for what are typically the most commonly heard tones in the music.
If you do need to tweak the source waves somewhat, you can either reauthor them and paste over the previous version in the wave bank, or you can make manual edits to how the regions play particular waves. The latter is useful when a wave is just slightly mispitched when played against another wave or when the levels of two recorded waves differ slightly. To override the wave's properties for this region, open the region's property window (by right-clicking and choosing Properties or, as with other property windows, by clicking on the region when the property window is already open).
Figure 2-11: A DLS region's property window. Here we've slightly adjusted the fine-tuning for BritePiano_G4 only as this region plays it back. If any other regions or instruments used the wave, their pitch would be unaffected by this change.
Note One common question is whether using overlapping regions on multiple layers could be used to more smoothly crossfade between waves. Unfortunately, all regions are triggered at the same velocity, so there is no easy way to make one region play more quietly than another for a given MIDI note. With some effort, you could potentially create several smaller regions over the transition that attenuate the source waves, gradually bringing their volume up (by overriding attenuation in the region's property window) to complete the crossfade.
And that's enough to create our basic instrument. If we wanted to, we could adjust any of a number of additional properties on the instrument's articulation, which is where much of the power of a DLS-2 synthesizer lies. The articulation allows you to specify volume and pitch envelopes, control a pair of low-frequency oscillators (LFOs), and set up low-pass filtering parameters for the instrument. You can even create per-region articulations, where a region has its own behavior that differs from other regions of the instrument. We'll just set up the instrument-wide articulation for this piano. Since our original sample already has most of the volume aspects of the envelope built in, we'll just add a release envelope. This means that when a MIDI note ends, the instrument will fade out over a period of time rather than cutting off immediately, much like an actual piano. By dragging the blue envelope point to the left in the Instrument Articulation frame, we have set the release to be .450 seconds, so this instrument's regions will fade out over roughly a half-second when a MIDI note ends.
Figure 2-12: To add any per-region articulations (presumably different from the global instrument articulation), right-click on the region and choose Insert Articulation List.
Our basic piano instrument is now complete. We could open up the instrument's property window (as always, by right-clicking on it and choosing Properties) and give it a better name than Instrument, such as MyPiano. We then repeat the process for other instruments in our collection. Again, remember that DLS Collections will be loaded into memory when a Segment is using them, so you'll want to budget their size according to available memory on the system on which they will be played.
Stereo Waves in DLS Collections
One unfortunate omission from the DLS-2 specification is that stereo waves are not supported for regions. DirectMusic Producer works around this by using DLS-2's aforementioned support for multiple layers. When a stereo wave is used in a DLS region, DirectMusic Producer creates a hidden second region, separates the two source channels out, and plays the mono pair of channels in sync on the two regions' collection.
Figure 2-13: The top single stereo region is actually stored by DirectMusic Producer as something closer to the bottom pair of mono regions.
All of this is transparent to the composer (and to any content that uses the DLS instrument) — they can use stereo waves the same as mono waves without having to do anything differently. But it does have several impacts on content authoring. The primary implication of DLS-2 not supporting stereo waves is that stereo waves cannot be compressed in DirectMusic Producer. Otherwise, DirectMusic would have to somehow figure out how to separate the left and the right channels to place them in separate mono regions. If you do intend to use stereo waves and they must be compressed, you must author them as pairs of mono waves, create regions on two layers, and then set those regions' multichannel properties to specify the channels and phase locking of the waves (so they are guaranteed to always play in sync).
Figure 2-14: Making our piano's right channel wave play in sync with the left channel and on the right speaker. Both channels would be set to the same Phase Group ID, and one of them should have the Master check box checked.
Using DLS Collections in Segments
Now that we've assembled our DLS Collection, our Segment needs to use that collection's patch changes in order to play its instruments. Remember that patch change information is stored in a Band in the Band Track. If we created a MIDI-based Segment from scratch, we would insert a Band Track (right-click in the Segment Designer window and choose Add Tracks…), and then insert a Band into the Track (by clicking in the first measure and hitting the Insert key).
However, let's assume for our very first piece of music that we just imported a MIDI file (from the File menu, choose Import File into Project…, then MIDI File as Segment…). In this case, we already have a Band created for us containing the patch changes from the original MIDI file.
We now want to edit this Band to use our new DLS instruments (rather than the General MIDI ones, or worse, instruments that don't exist).
Figure 2-15: Double-click the Band in the Band Track to open the Band Editor.
Figure 2-16: The Band Editor window.
Remember that in addition to patch change information, you can set up the initial volume and pan for each channel. The grid at the right allows you to do this graphically (vertical corresponding to volume, horizontal corresponding to left-right pan) by grabbing a channel's square and dragging it. Alternatively, you can set up each Performance channel by double-clicking on it (or right-clicking and choosing Properties).
The properties window is where you set the instrument properties for the channel.
Figure 2-17: The Band Properties dialog box.
An instrument's priority allows DirectMusic to know which voices are the most important to keep playing in the event you run out of voices. Volume and Pan are once again the same functionality seen in the Band Editor's right grid. Transpose lets you transpose all notes played onto the channel ("Oct" is for octaves and "Int" is for intervals within an octave). PB Range lets you control how far the wave will be bent by Pitch Bend MIDI controllers.
Range is a somewhat interesting option. It instructs DirectMusic that we will only play notes within a certain range, and therefore that we only need the regions from this instrument that are in that range. While this can cut down on the number of DLS instrument regions that are in memory (and thus possibly the size of wave data in memory), it does mean that notes played outside of this range will fall silent. Because transposition, chord usage, or other interactive music features might cause our content to play in a wider-than-authored range, Range is generally not used (and thus left unchecked).
Getting back to the task at hand, this channel is currently using General MIDI patch 89 (remember that the other two aspects of an instrument ID, the MSB and LSB in the DLS Collection, are typically zero for General MIDI collections). We want it to instead play our new instrument, so we click on the button displaying the name of the current instrument, choose Other DLS… from the drop-down that appears (the other options are all of the instruments from gm.dls), and select our DLS Collection and instrument to use.
Figure 2-18: The Choose DLS Instrument dialog box.
If we now play our Segment, Performance channel one will use our newly created piano instrument.
Authoring Flexible Content for Dynamic Playback
Playing back long MIDI sequences isn't going to be particularly interactive, so we might want to take this opportunity to consider alternative solutions to strictly linear scores. For instance, you might want to "chop up" your score into smaller pieces of music that can smoothly flow into each other. Authoring music in this kind of segmented (no pun intended) manner can be as simple as creating a bunch of short looping clips, each of which follows the same progression, perhaps deviating in terms of instrumentation and ornamentation. Alternatively, the various sections could be much more distinct, and you could author transitional pieces of audio content that help "bridge" different areas of the music. In every case, you need to consider the balance between truly "interactive" audio (that is, music that can jump to a different emotional state quickly) and musical concepts such as form, melodic line, and so on. That is, how can a piece of music quickly and constantly switch emotional levels and still sound musically convincing? Finding a balance here is perhaps one of the most significant challenges in creating dynamic audio content,
Transitions
Now that we've got basic linear pieces of music, let's discuss moving from one piece of linear music to another. How do we move from one piece of music (the "source") to the next (the "destination")? DirectMusic provides an abundance of choices in how such transitions occur. You can specify a Segment's default behavior in its property page (right-click on the Segment) on the Boundary tab. On the positive side, using these settings means you don't need to tell a programmer how your musical components should transition between each other. On the negative side, this locks a Segment into one specific transition behavior, which is often not desired; it's quite common to use different transition types based on both the Segment that we're coming from and the Segment that we're going to. Overriding can be a chore, particularly when using DirectX Audio Scripting. More often than not, audio producers end up having to communicate their ideas for transitions directly to the project's programmer for implementation. This is of course if you're working on a game. If you are creating stand-alone music for a DirectMusic player, stick to what you've got in terms of transition types!
Looking over the boundary options, there are quite a few that are fairly straightforward — Immediate, Grid (regarding the tempo grid; remember that the size of a grid can be specified in the Time Signature Track), Beat, and Measure. End of Segment and End of Segment Queue are also fairly easily understood — the former starting this Segment when the previous one ends and the latter starting this Segment after any other Segments that were queued up to play when the current Segment ended.
Marker transitions introduce us to another Track type that we can add to an existing Segment, aptly named the Marker Track. After adding a Marker Track to a Segment, an audio producer can add markers (also known as exit switch points) wherever it is deemed appropriate in the Segment, which will postpone any requested transition until a specific "legal" point is reached. Just to clarify, the transition is specified on the Segment that we're going to, even though the boundary used will be based on the currently playing Segment that we're coming from.
Transitions do not have to occur directly from one piece of music to another. The content author can create specific (typically short) pieces of music that make the transition smoother, often "ramping down" or "ramping up" the emotional intensity, altering the tempo, or blending elements of the two Segments. A programmer or scripter can then specify a transition Segment be used in conjunction with any of the boundaries discussed above.
As a more advanced behavior, when using Style-based playback, the programmer or scripter can specify particular patterns from the source and/or destination Segments (for instance, End and Intro, which plays an ending pattern from the source Segment's Style and an intro pattern from the destination Segment's Style) to similarly make the transition more convincing. We cover these embellishment patterns more in depth when we get into Style-based playback in Chapter 4.
By default, transitions go from the next legal boundary in the source Segment to the beginning of the destination Segment. There are, of course, cases where we might want the destination Segment to start from a position other than the beginning. Most commonly, we could intend for the destination Segment to start playing from the same relative position, picking up at the same measure and beat as the source Segment that it is replacing.
For this kind of transition, alignment can be used, again in conjunction with the various boundaries and transition Segment options above. Align to Segment does exactly what we outlined above. As an example, if the source Segment is at bar ten beat two, the destination Segment will start playing at its own bar ten beat two. Align to Barline looks only at the beat in the source Segment and can be used to help maintain downbeat relationships. Using the same example as above, the destination Segment would start in its first bar but at beat two to align to barline with the source Segment. As another technique for starting a Segment at a position other than its beginning, enter switch points (specified in the "lower half" of Marker Tracks) can be used. In this case, whenever a transition is intended to occur, the destination Segment will actually force the source Segment to keep playing until an appropriate enter switch point is reached in the destination Segment. Alignment generally provides sufficiently satisfactory transitions with less confusion, so enter switch points are rarely used. Remember in both cases that the limitations outlined earlier for pause/resume apply here; starting a Segment from a position other than the beginning will not pick up any already sustained MIDI notes, though it will pick up waves in Wave Tracks, as well as subsequent MIDI notes.
Scott Selfon
Okay, let's get cracking. The first thing we want to do is be able to play sound. We add variability, dynamism, and the rest later. Let's focus on playing back a wave file.
Note At this point, if you have not already, you must install DirectX 9.0 as well as DirectMusic Producer, which are available on the companion CD.
Waves in Wave Tracks
To play back a wave file in DirectMusic, you'll create a Segment out of it. As already mentioned, the DirectMusic Segment file (.sgt) is the basic unit of DirectMusic production. Segments are built from one or more of DirectMusic's various track types, which can make sound (via stand-alone waves and/or DLS instrument triggering) or modify performances (tempo, chord progression, intensity level, etc.). You can create a Segment out of a wave file in DirectMusic Producer. Run DirectMusic Producer and go to File>Import File into Project>Wave File as Segment…. Open a wave file into the program. Building a Segment from a wave file, we get our first look at one of the basic track types in DirectMusic — the Wave Track. Beyond the 32 variation buttons (which we cover in Chapter 3), Wave Tracks play along the sequencer timeline with other Wave Tracks and also with MIDI sequences.
If you look at the size of the Segment file, it is much smaller than the wave file. Here is an important early lesson in content delivery: Segments will know where the wave and sample files that it needs to play are but do not store those files as part of itself. There are several reasons to do this; if several Segments use the same wave data, you do not have to worry about having two copies of it in memory. In addition, if you later want to go back and edit that wave data, you do not have to worry about copying it into several different places. That said, there are various reasons that you might instead want to embed the wave data within the Segment itself — for instance, for the convenience of only having to deliver a single file or file load time considerations.
Files with the extension ".**p" are design-time files. Design-time files are used in DirectMusic Producer for editing purposes and contain information not necessary for run-time use — for instance, a Segment used in a game does not need to include information on what size and position to open editing windows. In addition, design-time files always reference the content they use, even if you specify that content should be embedded. For these reasons, when content is meant to be integrated into a game or a special player, you should save the Segment as a run-time file (either via the per-file right-click menu Runtime Save As… option or the global Runtime Save All Files option from the File menu). When Segments are run-time saved, you will see the .sgt extension, and wave files will similarly have the more expected .wav extension. So to summarize, be sure to save your Segments as design-time (.sgp) files while you are working on them and as run-time files (.sgt) when they are finished and ready for distribution. For more details on content delivery, there is a white paper available on the Microsoft Developer Network web site (msdn.microsoft.com) called "Delivering the Goods: Microsoft DirectMusic File Management Tips."
Streaming Versus In-Memory Playback
A minute-long 44.1 kHz 16-bit stereo wave file is 10MB. 10MB isn't a big deal when you run a stand-alone DirectMusic file on a contemporary computer, but it is a very large file in the world of game audio. Do not forget that the rest of a game's resources need to reside in memory as well. If the game is written to run on a system with as little as 64MB of RAM, you are already in way over your head. Console games are even more unforgiving. Consider yourself lucky if you get 4MB for your entire sound budget! You can use audio streaming to alleviate these restrictions. Streaming is a technique that works a lot like a cassette player. In a cassette player, audio data is moved across the play head a little bit at a time. The play head reads the data as it comes and plays the appropriate sound. With streaming, a small area in memory called a buffer is created. Wave files are moved, bit by bit, through the buffer and read by DirectMusic. The CD player in your PC uses streaming for playback. If it weren't for streaming, you'd have to load the entire music file into your computer's RAM, which in most cases simply isn't an option.
DirectMusic uses the following rule for streaming: The wave streams with 500 msec readahead if it is more than five seconds long. If it is shorter than five seconds, it resides and plays from memory (RAM). Readahead determines the size of the buffer. For 500 msec, our 44.1 kHz 16-bit stereo wave file will use 88.2KB of memory (.5 sec x 44100 samples/sec x 2 bytes/sample x 2 channels), a big difference when compared to 10MB! Memory usage is reduced by a factor of more than 100! You can override this behavior, choosing to load all wave data to memory, or specify a different readahead value in the Compression/Streaming tab of a wave's property page in DirectMusic Producer. To get to any Track's property page, simply right-click on it in the Track window.
Tracks and Parts and Layers, Oh My
Before we move any further, the concept of Tracks and parts can use a bit of sorting out. When we created our Segment from a wave file, it consisted of a single Track, a Wave Track, with a single part (by default on pchannel 1). Each part specifies its own variation behavior, what pchannel it is played on, and, in the case of Wave Tracks, volume for that part as well. By comparison, Tracks specify big picture (or "global") behaviors such as clock-time, as well as more exotic settings like multiple, concurrent Track groups. If you open the property page for a Track or part, you'll notice the separate tabs with their own settings — the Track properties actually includes both the Track Group Assignments and Flags tabs in addition to the actual Track properties tab.
Remember that waves in Wave Track parts never conflict with DLS Collections on a pchannel; you can play as many waves as you want simultaneously and still play a DLS Collection instrument on the same pchannel without any problem. The reason a Wave Track part can be assigned to a pchannel is that MIDI controllers can still be used to drive wave playback (for instance, pan on a mono wave, pitch bend, volume, etc.). The pchannel that a Wave Track is assigned to can be altered in the wave part's property page.
This brings us to the concept of layers, or the various lettered rows (a, b, c, etc.) that you see displayed in DirectMusic Producer for a single Wave Track part. As mentioned, you can play as many waves at one time as you wish. Therefore, layers are purely an aid for placing waves against the Segment timeline, rather than having many waves overlap on a tiny area of the screen and not be able to easily edit them (or tell which wave started and finished when). Waves can be placed on different layers within a part for easier legibility. All layers are equal, are played simultaneously, and do not take any additional resources (processor power or memory) when played.
Figure 2-1: Multiple waves on Wave Tracks are split into "layers" so you can see their beginnings and endings more easily.
As a last bit of terminology for now, certain parts are actually subdivided even further into strips. In particular, the parts for MIDI-supporting Tracks (Pattern Tracks and Sequence Tracks) have separate strips for note information and continuous controller information. Pattern Tracks also have an additional variation switch point strip, which we cover in Chapter 3 when we discuss variations.
MIDI and DLS Collections
Creating a DirectMusic Segment from a piece of MIDI music is simple. In DirectMusic Producer, follow File>Import File into Project>MIDI File as Segment…. This creates a DirectMusic Segment from a MIDI file, importing note, MIDI controller, and other pertinent data along the way. Let's examine some new Track types related to MIDI:
Tempo Track: Specifies the current tempo for the piece of music. You can override this by playing a primary or controlling secondary Segment with its own tempo settings.
Time Signature Track: Sets the time signature for the piece. Use this to track where measures fall as well as how many subdivisions (grids) to give each beat.
Chord Track: Specifies the key as well as specific chord progressions for a piece of music. Typically, for an imported MIDI file, this will just indicateaCmajor chord. DirectMusic Producer does not attempt to analyze the imported MIDI file for chordal information.
Sequence Track: A sequence is what its name implies. Sequence Tracks house MIDI sequences. This is where the majority of MIDI information is imported. Notice that as with Wave Tracks, Sequence Tracks can (and typically do) consist of multiple parts. By default, each MIDI channel is brought in as a separate part on the corresponding pchannel. In addition, each part can contain its own continuous controller information. Unlike Pattern Tracks (more on these in Chapter 3), Sequence Tracks are linear, non-variable sequences; they play the same every time, and they do not respond to chord changes (though they do respond to tempo changes).
Figure 2-2: A Segment with a Sequence Track consisting of multiple instruments.
Band Track: Bands are how Performance channels refer to DLS instrument collections. The Band Track is an area where initial pan, volume, and patch change settings for Tracks are stored. This is often an area of confusion, as you can also have volume and pan controller data within Sequence Track (or Pattern Track) parts. The Band Track typically just has a single Band with the initial settings. Subsequent volume changes are typically created as continuous MIDI controller events in Sequence or Pattern tracks. Continuous controller events, unlike band settings, allow you to easily sweep values over time (for example, to fade a track in or out). If any patch changes occur during performance, another Band can be placed in the Band Track, though the elimination of MIDI channel limitations means there is often little reason not to just place each instrument on its own unique and unchanging channel.
Building DLS Collections
Our Segment is now ready for playback. Of course, we're assuming that the piece of music will be played using a preauthored DLS Collection, such as the gm.dls collection that comes with Windows machines. Otherwise, it will be trying to play instruments that don't exist on the end user's machine, and these instruments would be played silently. If we wanted to use our own instruments, we would want to build one or more DLS Collections. DirectMusic Producer provides this authoring ability in DLS Designer. Alternatively, the DLS-2 format is a widespread standard, and there are several tools out there for converting from other common wavetable synthesizer formats to the DLS format.
Creating DLS Collections is often one of the more challenging tasks when you decide to create real-time rendered (versus streamed prerendered) audio. Remember that unlike streamed audio, your DLS Collection will occupy system memory (RAM), which is typically one of the most precious commodities for an application. For this reason, you'll want to create collections that get you the most bang for your buck in terms of memory versus instrument range and quality.
Let's create our first DLS Collection. From the File menu, select New, and from the dialog that appears, choose DLS Collection and hit OK. We're presented with a project tree entry for our DLS Collection (using the design-time .dlp file extension), which has two subfolders, Instruments and Waves.
Figure 2-3: An empty DLS Collection in the project tree.
DLS Collections are composed of waves (effectively, ordinary wave files) and instruments, which consist of instructions for how waves should map to MIDI notes, along with a fairly sophisticated set of audio-processing features. Let's add a few waves for our first instrument, a piano. We can drag our waves right into the Waves folder from a Windows Explorer window or right-click on the Waves folder and choose Insert Wave… from the menu that appears.
Figure 2-4: We've inserted four wave files into our DLS Collection.
We can now adjust the properties for these waves by right-clicking on them and choosing Properties to bring up their property window. The most important settings to note are going to be Root Note, loop points (both in the Wave tab), and compression settings (in the Compression/Streaming tab). We'll return to compression a bit later. Root Note specifies the base note that this wave corresponds to, also known in other wave authoring programs as the "unity pitch" or "source note." For our above piano sounds, determining the root note was made easier by including the root note information right in the wave file names. DirectMusic Producer will also automatically use any root note information stored with the wave file, which some wave authoring tools will provide. Otherwise, we can adjust the root note manually from the property page, using either the scroll arrows or by playing the appropriate note on a MIDI keyboard.
Figure 2-5: Setting the proper root note for our BritePiano_C4 wave.
This particular piano wave doesn't loop, so we don't need to worry about the loop settings. Again, if loop settings had been set on the source wave file, DirectMusic Producer would automatically use that information to set loop points.
DirectMusic Producer provides some basic editing features on waves in the Wave Editor, which can be opened by double-clicking any wave in the Waves folder (or indeed, any separate wave file you've inserted into your DirectMusic project).
Figure 2-6: The Wave Editor. You can specify whether waveform selections (made by clicking and dragging) should snap to zero crossings via Snap To Zero, and you can specify a loop point by selecting Set Loop From Selection.
The Wave Editor window supports clipboard functions, so you can copy to and paste from other wave editing applications if you wish. You can also try out any loop points you've set (or just hear the wave you've created) by playing the wave. As with Segment playback, you can use the Play button from the Transport Controls toolbar, or use the Spacebar as a shortcut to audition your wave. Note that you can only start playing your waves from the beginning, regardless of where you position the play cursor, as DLS waves cannot be started from an offset.
Now that we've set up our waves, let's add them to a DLS instrument so we can play them via MIDI notes. To create an instrument, right-click on the Instruments folder and choose Insert Instrument.
Figure 2-7: Creating an instrument.
You'll notice that each instrument is assigned its own instrument ID, a unique combination of three values (MSB, LSB, and Patch) that allows for more than 2 million instruments — a bit more freeing than the traditional 128 MIDI patch changes. DirectMusic Producer will make sure that all loaded DLS Collection instruments use unique instrument IDs, but you should take care if you author your DLS Collections in different projects to make sure that they don't conflict. Otherwise, if two instruments with identical IDs are loaded at the same time, DirectMusic will have no way of knowing which one you want to use. The General MIDI DLS Collection (gm.dls) that is found on all Windows machines uses 0,0,0 through 0,0,128, so our new instrument probably defaulted to instrument ID (0,1,0).
Let's open up the Instrument editor by double-clicking on the instrument in the project tree.
Figure 2-8: The Instrument editor.
There are lots of options here that really demonstrate the power of a DLS-2 synthesizer, but for now let's just start with our basic instrument. The area above the piano keyboard graphic is where we define which wave or waves should be played for a given note. Each wave assigned to a range of notes is called a DLS region. As you can see, our new instrument defaults to a single region, which plays the wave BritePiano_C5, over the entire range of MIDI notes. The piano keyboard note with a black dot on it (C5) indicates the root note for the region. Remember that the root note will play the wave as it was authored, notes above the root note will pitch the wave up, and notes below the root note will pitch the wave down.
Looking along the left edge of our region, you can see the labels DLS1, 1, 2, and so on. These labels identify the various layers of this instrument. Layers allow you to create overlapping regions, one of the nice features of the DLS-2 specification. One of the more common uses is for adding multiple velocity versions of a wave, where as the note is struck harder, the instrument will switch to a different wave file. This is particularly effective for percussion instruments. Notice that each region has a velocity range that can be set in the Region frame. Multiple layers also mean that a single MIDI note could trigger a much more complex instrument composed of several simultaneously played wave files. Remember that regions on a single layer cannot overlap. The DLS1 layer is the single layer that was supported in the original DLS-1 specification, and is generally still used to author any single-layer DLS instruments. For this simple instrument, we won't worry about using multiple layers for the moment.
Our first step is to create regions for the rest of our piano waves. We want to resize that existing region so there is room on the layer for the other waves (that, and we probably don't want to try to play this wave over the entire range of the MIDI keyboard!). If you move the mouse over either edge of the region, you'll see that it turns into a "resize" icon, and you can then drag in the outer boundaries of the region.
Figure 2-9: Resizing a region with the mouse. Alternatively, we could change the Note Range input boxes in the Region frame.
By the way, this is similar to the functionality for note resizing (changing start time and duration) in the MIDI-style Tracks (Pattern Tracks and Sequence Tracks). There are other similarities between region editing and note editing — the selected region turns red, just as notes do. The layer we are currently working with turns yellow, just as the current Track does in the Segment Designer window. And just as you can insert notes by holding the Ctrl key while clicking, you can create a new region in the same way.
You can resize the region as long as the mouse is held down and dragged (or, as before, resize the region by grabbing a side edge of it). As an alternative to drawing a region with the mouse, you can right-click and choose Insert Region from the menu that appears.
Every region that we create defaults to use the same wave that the first region did. So we'll want to choose a more appropriate wave from the drop-down aptly labeled Wave. Let's assign BritePiano_E5 to this new region we've created.
We repeat the process for the rest of our waves to build up our instrument. Notice as you click on each region that you can view its particular properties for wave, range, root note, etc., in the Region frame.
Figure 2-10: The completed regions for our piano instrument.
There are several potential issues to discuss here. First, should all instruments span the entire keyboard? On the plus side, the instrument would make a sound no matter what note was played. On the minus side, when the source wave is extremely repitched (as BritePiano_C4 is in the downward direction and BritePiano_E5 is in the upward direction), it can become almost unrecognizable, dramatically changing timbre and quality. Most composers opt for the latter, if only to know whether their music ever goes significantly out of an acceptable range for their instruments. If it does, the music can be adjusted (or the DLS instrument expanded to include additional regions).
A second question is how far above and/or below the root note a region should span. That is, does the wave maintain the quality of the source instrument more as it is pitched up or down? This can vary significantly from wave to wave (and the aesthetic opinions of composers also differ quite a bit). For the above example, we did a bit of both — the bottom three regions encircle notes both above and below the root note, while the highest piano region (our BritePiano_ E5 wave) extends upward from the root note. How far you can "stretch" a note can vary quite significantly based on the kind of instrument and the source waves.
Once you've created your instrument, you can try auditioning it in several ways. An attached MIDI keyboard will allow you to trigger notes (and indeed, small red dots will pop up on the keyboard graphic as notes are played). You can also click on the "notes" of the keyboard graphic to trigger the note. The Audition Options menu allows you to set the velocity of this mouse click and choose whether to audition all layers or only the selected one. (In the case of our example, we only have one layer, so the two options don't matter.)
Try out the piano instrument, paying particular attention to the transitions between regions. These are often difficult to match smoothly, as the pitch shifting of the waves begins to alter their characteristics and the instruments can begin to develop musical "breaks" where the tone quality changes significantly. If you are writing a purely tonal piece of music, it is often useful to author your DLS Collections such that the tonic and dominant tones (the scale root and the 5th degree) are the root notes of your regions. That way, the "best" version of each wave (not repitched) will be played for what are typically the most commonly heard tones in the music.
If you do need to tweak the source waves somewhat, you can either reauthor them and paste over the previous version in the wave bank, or you can make manual edits to how the regions play particular waves. The latter is useful when a wave is just slightly mispitched when played against another wave or when the levels of two recorded waves differ slightly. To override the wave's properties for this region, open the region's property window (by right-clicking and choosing Properties or, as with other property windows, by clicking on the region when the property window is already open).
Figure 2-11: A DLS region's property window. Here we've slightly adjusted the fine-tuning for BritePiano_G4 only as this region plays it back. If any other regions or instruments used the wave, their pitch would be unaffected by this change.
Note One common question is whether using overlapping regions on multiple layers could be used to more smoothly crossfade between waves. Unfortunately, all regions are triggered at the same velocity, so there is no easy way to make one region play more quietly than another for a given MIDI note. With some effort, you could potentially create several smaller regions over the transition that attenuate the source waves, gradually bringing their volume up (by overriding attenuation in the region's property window) to complete the crossfade.
And that's enough to create our basic instrument. If we wanted to, we could adjust any of a number of additional properties on the instrument's articulation, which is where much of the power of a DLS-2 synthesizer lies. The articulation allows you to specify volume and pitch envelopes, control a pair of low-frequency oscillators (LFOs), and set up low-pass filtering parameters for the instrument. You can even create per-region articulations, where a region has its own behavior that differs from other regions of the instrument. We'll just set up the instrument-wide articulation for this piano. Since our original sample already has most of the volume aspects of the envelope built in, we'll just add a release envelope. This means that when a MIDI note ends, the instrument will fade out over a period of time rather than cutting off immediately, much like an actual piano. By dragging the blue envelope point to the left in the Instrument Articulation frame, we have set the release to be .450 seconds, so this instrument's regions will fade out over roughly a half-second when a MIDI note ends.
Figure 2-12: To add any per-region articulations (presumably different from the global instrument articulation), right-click on the region and choose Insert Articulation List.
Our basic piano instrument is now complete. We could open up the instrument's property window (as always, by right-clicking on it and choosing Properties) and give it a better name than Instrument, such as MyPiano. We then repeat the process for other instruments in our collection. Again, remember that DLS Collections will be loaded into memory when a Segment is using them, so you'll want to budget their size according to available memory on the system on which they will be played.
Stereo Waves in DLS Collections
One unfortunate omission from the DLS-2 specification is that stereo waves are not supported for regions. DirectMusic Producer works around this by using DLS-2's aforementioned support for multiple layers. When a stereo wave is used in a DLS region, DirectMusic Producer creates a hidden second region, separates the two source channels out, and plays the mono pair of channels in sync on the two regions' collection.
Figure 2-13: The top single stereo region is actually stored by DirectMusic Producer as something closer to the bottom pair of mono regions.
All of this is transparent to the composer (and to any content that uses the DLS instrument) — they can use stereo waves the same as mono waves without having to do anything differently. But it does have several impacts on content authoring. The primary implication of DLS-2 not supporting stereo waves is that stereo waves cannot be compressed in DirectMusic Producer. Otherwise, DirectMusic would have to somehow figure out how to separate the left and the right channels to place them in separate mono regions. If you do intend to use stereo waves and they must be compressed, you must author them as pairs of mono waves, create regions on two layers, and then set those regions' multichannel properties to specify the channels and phase locking of the waves (so they are guaranteed to always play in sync).
Figure 2-14: Making our piano's right channel wave play in sync with the left channel and on the right speaker. Both channels would be set to the same Phase Group ID, and one of them should have the Master check box checked.
Using DLS Collections in Segments
Now that we've assembled our DLS Collection, our Segment needs to use that collection's patch changes in order to play its instruments. Remember that patch change information is stored in a Band in the Band Track. If we created a MIDI-based Segment from scratch, we would insert a Band Track (right-click in the Segment Designer window and choose Add Tracks…), and then insert a Band into the Track (by clicking in the first measure and hitting the Insert key).
However, let's assume for our very first piece of music that we just imported a MIDI file (from the File menu, choose Import File into Project…, then MIDI File as Segment…). In this case, we already have a Band created for us containing the patch changes from the original MIDI file.
We now want to edit this Band to use our new DLS instruments (rather than the General MIDI ones, or worse, instruments that don't exist).
Figure 2-15: Double-click the Band in the Band Track to open the Band Editor.
Figure 2-16: The Band Editor window.
Remember that in addition to patch change information, you can set up the initial volume and pan for each channel. The grid at the right allows you to do this graphically (vertical corresponding to volume, horizontal corresponding to left-right pan) by grabbing a channel's square and dragging it. Alternatively, you can set up each Performance channel by double-clicking on it (or right-clicking and choosing Properties).
The properties window is where you set the instrument properties for the channel.
Figure 2-17: The Band Properties dialog box.
An instrument's priority allows DirectMusic to know which voices are the most important to keep playing in the event you run out of voices. Volume and Pan are once again the same functionality seen in the Band Editor's right grid. Transpose lets you transpose all notes played onto the channel ("Oct" is for octaves and "Int" is for intervals within an octave). PB Range lets you control how far the wave will be bent by Pitch Bend MIDI controllers.
Range is a somewhat interesting option. It instructs DirectMusic that we will only play notes within a certain range, and therefore that we only need the regions from this instrument that are in that range. While this can cut down on the number of DLS instrument regions that are in memory (and thus possibly the size of wave data in memory), it does mean that notes played outside of this range will fall silent. Because transposition, chord usage, or other interactive music features might cause our content to play in a wider-than-authored range, Range is generally not used (and thus left unchecked).
Getting back to the task at hand, this channel is currently using General MIDI patch 89 (remember that the other two aspects of an instrument ID, the MSB and LSB in the DLS Collection, are typically zero for General MIDI collections). We want it to instead play our new instrument, so we click on the button displaying the name of the current instrument, choose Other DLS… from the drop-down that appears (the other options are all of the instruments from gm.dls), and select our DLS Collection and instrument to use.
Figure 2-18: The Choose DLS Instrument dialog box.
If we now play our Segment, Performance channel one will use our newly created piano instrument.
Authoring Flexible Content for Dynamic Playback
Playing back long MIDI sequences isn't going to be particularly interactive, so we might want to take this opportunity to consider alternative solutions to strictly linear scores. For instance, you might want to "chop up" your score into smaller pieces of music that can smoothly flow into each other. Authoring music in this kind of segmented (no pun intended) manner can be as simple as creating a bunch of short looping clips, each of which follows the same progression, perhaps deviating in terms of instrumentation and ornamentation. Alternatively, the various sections could be much more distinct, and you could author transitional pieces of audio content that help "bridge" different areas of the music. In every case, you need to consider the balance between truly "interactive" audio (that is, music that can jump to a different emotional state quickly) and musical concepts such as form, melodic line, and so on. That is, how can a piece of music quickly and constantly switch emotional levels and still sound musically convincing? Finding a balance here is perhaps one of the most significant challenges in creating dynamic audio content,
Transitions
Now that we've got basic linear pieces of music, let's discuss moving from one piece of linear music to another. How do we move from one piece of music (the "source") to the next (the "destination")? DirectMusic provides an abundance of choices in how such transitions occur. You can specify a Segment's default behavior in its property page (right-click on the Segment) on the Boundary tab. On the positive side, using these settings means you don't need to tell a programmer how your musical components should transition between each other. On the negative side, this locks a Segment into one specific transition behavior, which is often not desired; it's quite common to use different transition types based on both the Segment that we're coming from and the Segment that we're going to. Overriding can be a chore, particularly when using DirectX Audio Scripting. More often than not, audio producers end up having to communicate their ideas for transitions directly to the project's programmer for implementation. This is of course if you're working on a game. If you are creating stand-alone music for a DirectMusic player, stick to what you've got in terms of transition types!
Looking over the boundary options, there are quite a few that are fairly straightforward — Immediate, Grid (regarding the tempo grid; remember that the size of a grid can be specified in the Time Signature Track), Beat, and Measure. End of Segment and End of Segment Queue are also fairly easily understood — the former starting this Segment when the previous one ends and the latter starting this Segment after any other Segments that were queued up to play when the current Segment ended.
Marker transitions introduce us to another Track type that we can add to an existing Segment, aptly named the Marker Track. After adding a Marker Track to a Segment, an audio producer can add markers (also known as exit switch points) wherever it is deemed appropriate in the Segment, which will postpone any requested transition until a specific "legal" point is reached. Just to clarify, the transition is specified on the Segment that we're going to, even though the boundary used will be based on the currently playing Segment that we're coming from.
Transitions do not have to occur directly from one piece of music to another. The content author can create specific (typically short) pieces of music that make the transition smoother, often "ramping down" or "ramping up" the emotional intensity, altering the tempo, or blending elements of the two Segments. A programmer or scripter can then specify a transition Segment be used in conjunction with any of the boundaries discussed above.
As a more advanced behavior, when using Style-based playback, the programmer or scripter can specify particular patterns from the source and/or destination Segments (for instance, End and Intro, which plays an ending pattern from the source Segment's Style and an intro pattern from the destination Segment's Style) to similarly make the transition more convincing. We cover these embellishment patterns more in depth when we get into Style-based playback in Chapter 4.
By default, transitions go from the next legal boundary in the source Segment to the beginning of the destination Segment. There are, of course, cases where we might want the destination Segment to start from a position other than the beginning. Most commonly, we could intend for the destination Segment to start playing from the same relative position, picking up at the same measure and beat as the source Segment that it is replacing.
For this kind of transition, alignment can be used, again in conjunction with the various boundaries and transition Segment options above. Align to Segment does exactly what we outlined above. As an example, if the source Segment is at bar ten beat two, the destination Segment will start playing at its own bar ten beat two. Align to Barline looks only at the beat in the source Segment and can be used to help maintain downbeat relationships. Using the same example as above, the destination Segment would start in its first bar but at beat two to align to barline with the source Segment. As another technique for starting a Segment at a position other than its beginning, enter switch points (specified in the "lower half" of Marker Tracks) can be used. In this case, whenever a transition is intended to occur, the destination Segment will actually force the source Segment to keep playing until an appropriate enter switch point is reached in the destination Segment. Alignment generally provides sufficiently satisfactory transitions with less confusion, so enter switch points are rarely used. Remember in both cases that the limitations outlined earlier for pause/resume apply here; starting a Segment from a position other than the beginning will not pick up any already sustained MIDI notes, though it will pick up waves in Wave Tracks, as well as subsequent MIDI notes.
Subscribe to:
Posts (Atom)