Interactive Music: Fmod and Unity 3D



The adaptive music GUI above, shows how music states are able to change dynamically based on user inputs. Although the music changes are triggered via simple button clicks these can easily be programmed to trigger during any game state. The purpose of this project was to prove functionality of the FMOD Designer events with Unity 3D and explore different methods of creating interactive music. In Spring 2011 I hope to deploy some of these methods as I work as part of 40+ member game studio.


The work-flow involved with this project included:

  • Music composition via Pro Tools and Ableton Live
  • Designing sound events with programmable parameters in FMOD Designer
  • Programming FMOD events in Unity
  • Creation of Unity GUI and standalone project executable



Music

Music tracks were developed such that each layer of instrumentation would provide an increase in the overall intensity. The starting ambient tracks are intended to be the unifying stems which provide continuity for all of the musical states. These ambient tracks vary in level but remain throughout each music state. Aside from a sub-mix of the final musical tension, all of the individual stems were exported individually in order to allow dynamic mixing to occur in FMOD.



FMOD

The initial plan was to generate a multichannel audio .wav file from the music stems created previously. In turn, only one layer on the FMOD event would need to be used. Unfortunately there was an issue with this particular version of FMOD Designer. As a workaround each of the music stems were rendered independently and imported as different event layers. Fades and levels were set via an effects layer while volume and seek parameters were adjusted to provide a natural progression from each music state. Randomized pitch and spawn counts were also given to ambient sound effects in order to provide more variation.




Unity

As a single FMOD event, initialization and playback in Unity was fairly straight forward. Unity's OnGUI function allows the FMOD music event's parameter value to be tied to button objects. In this case, the volume parameter and seek time need to be defined order for the correct transition to play. 5 state buttons were created along with a toggle button for seek time in order to instantly change states instead of gradually fading into them.