The Audio of Halo Wars – A victim of unambitious design

A couple of weeks before it’s official release I got the chance to visit Microsoft HQ in Sydney and play a preview build of the now defunct Ensemble studio’s Xbox RTS Halo Wars. I wrote up a little bit of what I thought for Kotaku Australia as part of the competition I won to go and play it. You can read what I said via the link, but essentially what I most wanted to express was that This is a game that knows what it wants to be, but I’m going to tell you that’s not exactly a good thing.

It’s the Halo Universe, even down to the splash of the plasma weapons. Edge Magazine noted in a review that appeared online briefly that the game is unashamedly a console one, and to paraphrase the reviewer, they said that “Halo Wars stares you down and says ‘This here’s console country, we do things a little differently’.” It’s strange because I get the feeling most games fail because of the opposite problem – often grafting in ancillary ideas from other games and genres in an effort to siphon their success. In development circles it’s known as ‘feature creep’.

So Halo Wars knows what it wants to be and we the audience already know what to expect. Kieron Gillen in his review of the game for Eurogamer said of the game that,

The biggest strength… is the fact that most people understand the Halo universe. It's not just the geek thrill of seeing a Scarab in action - it's that you understand what the Scarab means on the battlefield (trouble). We know which characters are best against tanks, and which are probably best in special vehicles.

Given that foreknowledge, Ensemble is (was?) hardly the studio to farm out a cheap sequel to – so where did they carve out a creative space of their own? Where did they leave their mark on the Halo universe? Unless you count inventing some new units, basically nowhere – and unless there’s another story behind the scenes, they seemed perfectly happy to do that. But, as we saw with the discussion surrounding FEAR 2, is “good enough” really good enough anymore?

Halo Wars is a nostalgia fest, make no mistake, and it would be easy to cynically view it as an attempt by Microsoft’s to cash in on one of their more successful pieces of videogame intellectual property. I’m giving them the benefit of the doubt, however, and I’m going to suggest that it was probably only partly an exercise in ensuring gamers maintain some interest in the Halo franchise. Ensemble’s efforts at channelling the Bungie vibe seem to fall well short of the mark, failing to spin as convincing and exciting a yarn as their predecessors (caveat: I haven’t finished the game yet). Ironically, one of the points Marty O’Donnell made in my interview with him was that some people are just better story tellers than others and for whatever else you can fault them, Bungie do often manage to tell a good story.

So Halo Wars rests in that uncomfortable half-way place between a creative and original expansion of the universe and outsourced publisher cash-in. I admit it would be self-aggrandizing to try and account for this by blaming it on the misappropriation of ‘classic’ Halo sounds in the game, but at the very least it is a symptom of the larger problem at hand.

Indeed, we saw it happening from the first trailer which was itself a studied exercise in comforting the confused and afraid fans. Halo is changing?! But I like Halo the way it is! Change is scary and confusing. Did the marketing executives focus test that first trailer to see whether that perception was actually correct? It is obvious that is what they were thinking, even electing to preserve as much of the ‘classic’ Halo as they could.

The look of Halo was obviously always going to change – that much was evident from both the first trailer and the fact that the game itself would change into an RTS. But the sounds, ‘Oh no’, the executives said with a million dollar smile, ‘The classic sounds we can preserve!’ In actuality, it was probably closer to someone saying ‘Hey, can we get the Halo plasma rifle sounds for our trailer? Sweet!’ and then they never considered changing out these place holders.

In my thesis which examined the sound and music of Halo 2, one of the main discoveries was an identifiable link between the music and sound effects – that a level of ‘musicality’ is imparted into the sound effects via common timbres and pitches. Many of the covenant weapon sounds were in fact produced using similar synthetic techniques, even the same instruments that were used in the production of music tracks which ended up picking up the role as musical themes for the covenant. These same sound effects have now been re-used in Halo Wars in much the same way that ‘stock’ library sounds would – and the result is an unnatural grafting along with the loss of their musical context.

‘Stock’ audio from bought libraries of recordings is often used in films (and games) in order to keep down recording costs and, occasionally, to artistically draw upon their sonic history. The ‘Wilhelm Scream’ doesn’t just sound like a scream for western listeners, if you’ve had even a moderate exposure to late 20^th Century films, you will have heard it enough times that it has become a scream. We hear the Wilhelm scream and know instantly what it means because we have heard it in context before. The sound itself has a history – we’ve heard it in Star Wars, we’ve heard it in Indiana Jones, and we’ve heard it in a million other movies since.

But stock sounds, like the Wilhelm scream, are designed to be memorable on their own, whereas the Halo sounds are made memorable by their connection to music and ‘level ambience’. The blurring of the lines between diegetic and non-diegetic sound and music in Halo 2 is a powerful technique that, in my mind, makes the sonic material all the more powerful. (See Chapter 4.4 of my thesis for a much more detailed discussion and an explanation of why it is such an effective technique.)

Halo Wars re-uses weapon sound effects from earlier Halo games with the aim of capitalizing on our history and experiences with them. We are encouraged to think back to those Halcyon days of Halo 3 and earlier, remembering our positive experiences with them. While it is a problem that these sounds don’t have nearly as much of a history as sounds such as the Wilhelm scream, lessening their historical impact, the more serious problem is that these sounds aren’t designed to exist without their musical and level-based context. Halo Wars carries the legacy sound effects of the Halo universe while unceremoniously dumping their musical counterparts. In losing their musical context, they have lose much of their attractiveness.

The argument for sonic ‘continuity’ between the Halo series is pre-eminently redundant – with every Bungie made Halo game, you get everything changing. From weapons and character designs to the visuals and the sounds, they are all subtly changed and revaluated. Even the way that sounds are mixed together live (what Marty says is actually half of what makes good game sound) is constantly being refined, polished and improved also. Bungie knows the value in re-imagining, redesigning and reconceptualising - even if we occasionally prefer aspects of the old to the new.

In the end, Ensemble opted for safety in their final game – and honestly it’s a struggle to blame them. Halo Wars is still horribly polished, and it’s still Halo… but it’s not really Ensemble saying ‘If we did Halo, what would we do differently?’ Instead it’s Ensemble kowtowing to the marketers, and more’s the pity. The sound, it’s really just a symptom of the wider problem – a disappointing lack of imagination.

Marty O'Donnell in Interview - Part 7, The Finale

In the final installment of my interview series with Marty O’Donnell, we talk about the effect that instrumentation has on musical themes and development, and I ask him one final question, what gets him out of bed in the mornings.

Marty: So it turned out that in Halo 2 there was sort of what developed into the covenant theme and I think it was the music that came in when the arbiter was introduced… and it tended to feel like the arbiters theme. But it felt more like ‘this is the somber alien, this has to do with the emotion of what is happening over there’ and that emotion worked several times through the game for me, so it wasn’t necessarily the covenant theme. I never developed the theme of the covenant, but it sorta turned out almost like that.

Ben: So you say you never wrote a covenant theme, but it seems to me like there’s definitely a 'covenant' type of instrumentation. Was that a deliberate thing?

M: It’s sorta like what you were saying before, there are a lot of different techniques and I tend to want to not be stuck on one, so sometimes it’s a musically thematic thing sometimes it’s just the colour of the instruments that are being used and then that’s what becomes sort of iconic throughout.

Lots of different thing work, and I don’t like to limit myself to only one way of working. It sort of self limits when you start really putting it together, you start saying, well I’m not going to do this even thought that could work, because this other thing has already become more interesting to me, so I’ll just narrow it down to just these three kind of approaches, and at the end of the day you look back and it almost looks like I planned the whole thing.

B: *laughs* Well it really does, I’m listening through it and going ‘hang on a minute there’s all these covenant sounds and they all sound like the covenant instrumentation!’

M: *laughs* Yeah it’s probably true, but it wasn’t something I planned ahead of time, it’s a little bit more organic. As you start throwing things together, certain things kinda work and stick so you stay with it, and you realize at the end maybe it was meant to be that way or… I don’t know how to put it, it’s never something I plan ahead of time, like ‘this is the way it will end up’, maybe it’s just my approach to composition, I tend to be more in the moment and say ‘hey this is working, I like this’ and then look back at what I’ve done and realize ‘oh there’s almost a plan here or there’s definitely a theme that’s building that I didn’t really know was going to be there ahead of time. But I sort of anticipate that it probably will be that way by the time I’m done.

B: Yes, I think that’s probably how I work as well. One of our lecturers at Uni always goes on about compositions being organic and having a natural development. I got into my degree as a singer, and so all my music training and knowledge was gained by ear, so I tend to evaluate whether it works on not by listening rather than dots on the page.

M: Yeah I think that’s great, that’s where it counts.

B: So I think that’s all of the questions I had planned to ask, except for one last question. I wanted to ask you what’s the most exciting thing that’s happening in music in games. What gets you out of bed in the morning to come to work after 10, 12 or 15 hours of crunch?

M: *laughs* Well that’s interesting…I would say, in general, the reason that I’m in the game business… I think it’s an artistic medium that’s still early in the development phase. And as a creative person who likes working in a place that is somewhat unexplored that is very exciting to me. There’s a lot of other places you can do music, but music in games still I think has just a huge unknown path before it. I have no idea where it’s really going to go. Games as a medium in general are still potentially in their infancy. I find it would have been really fascinating to have been a filmmaker in the 1920’s because it was so early on and things moved so amazing over the, from 29-39 it was just an amazing ten year period, and I think we’re sort of in that period. The mid 30’s in the film industry is where we are in the games industry. I think 5 years from now games are going to be vastly more improved, and improved as an artistic expression. And so I would encourage anybody who likes creating in a places that still has unexplored paths, I think games are probably the place to be.

Thanks again to Marty for being a part of my thesis and for being so generous with his time, and thanks to everyone for reading this rather epic series. Parts 1 through 7 can be found here.

Marty O'Donnell in Interview - Part 6

Sometimes, the most valuable lessons are learnt from the things that just don’t work as expected. In this, part 6 of the seemingly never ending interview saga that is my interview with Bungie’s Marty O’Donnell, he talks about the ‘misses’ in the music of Halo 2, working with music from outside Bungie, in particular the rock band ‘Incubus’, some of the reactions to the music and whether he likes to use musical ‘leitmotif’.

Ben: So can I change gears for a second and ask you if there were any areas in the music for Halo that you weren’t happy with? I think you mentioned in one interview that you probably didn’t quite the level of polish for the mix in Halo 2 that you wanted to. Was there anything that if you had another week or month you’d change?

Marty: The mix is probably the biggest thing. It’s really hard because you never know how somebody’s going to play. And the mix isn’t something you set, it’s that you allow the game to mix based upon what’s happening. So, if dialogue happens to come in, then whatever music is playing will be ducked 6db or whatever explosion is happening gets ducked… and playing around with all of those, attenuation and EQ and all the rest of it that we can do in real time and actually really testing it all out in as many ways a possible is just tediously hard to do, and we just never get enough time to do that… yeah, the mix is my biggest complaint.

One of my other complaints would probably be… just the amount of music that I would be able to compose after a level is completed and designed. I would rather be able to compose more music in response to what I’m playing rather than having to do it prior. My ideal would be that all the music that’s composed is composed for what I’m seeing and I never use another piece twice. Especially in Halo 2 I think you’re going to at least hear I think every piece of music probably is repeated somewhere in the game. But it’s not going to be repeated verbatim, because the system won’t let it, but it’s still the same piece. That is simply because we only have so much time and room to shove things on the disc. I would love to have it be a more original from beginning to end… I still think themes should return and be developed but I would rather have it truly be developed and be new than repeated.

And then the other thing for Halo 2, I experimented with using some music from some outside bands. I think that it’s a sort of mixed bag – stylistically it wasn’t as successful as I would have hoped.

B: I quite liked them, especially Incubus’s instrumental ‘The Odyssey’ pieces.

M: Oh good, well I liked that one too but it was tough. There were a couple other things I wasn’t as happy with and I actually wish I had more time to play with the incubus piece and actually work it in better than where I worked it.

B: So that wasn’t where you would have ideally put it in the game?

M: No, it was OK just I think in the long run, that particular level was a flying level and it was the only place I used it and I think it was a little bit soft and it didn’t’ quite do everything I was hoping it would do for that level.

And then I’ve had enough push-back from other people who were like “What the heck was that doing in there, that stuck out like a sore thumb!”

B: *laughs* Really? Wow.

M: Yeah, I’ve had people say that. But, y’know a lot of people, other people really liked it, and I liked it too, but there were some other things scattered throughout the game that I wasn’t necessarily as happy with.

And there’s always little choices that I’ve made, like here’s a piece I wrote and put in this section and think it’s great, then a month later after we ship the game I listen to it, ‘Wow, what was I thinking that really didn’t work’. That’s always going to happen.

B: On that development of themes idea… do you deliberately aim for that Leitmotif style?

M: Yeah, I’ve had people say ‘Hey here are the 5 main characters and here are the 6 locations and we think it would be great to have a theme for each location and a theme for each character and I basically say, that’s not me that’s not the composer that I want to be. I’m not writing Peter and the Wolf, as much as I love Peter & the Wolf… I’m not Wagner, but I think Wagner did some things that I think were kinda cool…

So it’s not like ‘Here’s a theme and it comes in every time this guy comes [on the screen]’. Sometimes by having certain themes that are kind of associated with some characters or even the emotion a character might feel, what happens is… this is just me, my personal opinion… if you say here’s the heroic Master Chief theme and every time Master Chief does something you play this theme, it’s like, well, how does that work when Master Chief is the player, and the player is really freaked out or scared or something – that theme just doesn’t work. I like ‘emotional leitmotifs’, so to speak. So it’s like, ‘Here’s the love theme, here’s the heroically successful theme, here’s the scary theme’…and then it doesn’t matter who the characters are if they’re going through that emotion I can bring that in.

These things tend to develop almost on their own – you realize… this music worked really worked well over here and this is the emotion that was happening, so I can do that again over here because that same kind of emotion is happening.

It seems like it ties together, and it seems like a thematic thing but it’s almost like, ‘Wow, isn’t that great that that works?’ Anyway that’s my approach if that makes sense.

In part 7, the final installment in the series, Marty Talks a bit more about his ‘emotional leitmotif’ approach, the affect of instrumentation on his music and finally he answers the question ‘What gets you out of bed in the morning’.

Marty O'Donnell in Interview - Part 5

In part five of my interview series with Marty O’Donnell, I ask about music in multiplayer game modes, to which Marty shares a few tidbits about what he’s currently experimenting with. He also talks about music for MMO’s and gets onto the topic of musical genre and appropriateness for videogames upon which he expresses his desire for game music to do more than emulate the successes of the past.

Ben: So I want to ask you about multiplayer, and the fact that there’s no music. Was that decision purely because it would get boring, or get in the way, or is it a reflection on the way you think about the role of music in a game – is it different when you’re telling a story in the campaign to, in multiplayer, where it’s almost like an electronic sport, rather than a story?

Marty: Yeah, so here’s my philosophy about that, and it’s not necessarily what we’re always going to do, as a matter of fact even as we speak I’m thinking about certain multiplayer modes where music might have a place. I’m a little bit suspicious that it’s ever going to be all that effective because I still believe strongly that music essentially tells basically linear stories, that music has a beginning, a middle and an end, and music creates emotional enhancements that work best when you have reasons for music to exist.

If people just want to say, hey when I’m playing Halo I just want to be really revved up, I’m like ‘sure, go ahead, put some Metallica on while you play, I don’t care, that’s great. You have every right to put on any kind of piece of music that just gets you in a mood want. There’s not enough hooks in a multiplayer game to make me feel like music will do anything but probably get annoying or boring or repetitive.

Sort of like, every time you win you get the big winning music. Well that’s going to be annoying because when you’re losing you’re probably annoyed because you hear the losing music. Or you’re winning and it’s like, well here I am hearing that same winning music again. Even if you have lots of variations it just doesn’t seem like… I never thought it was a huge payoff.

B: You could end up playing 300 or so games…

M: Well you play it so much, right… And the other thing is that I think there’s a difference in the player’s attitude about why they’re playing. When you’re playing the campaign…you are becoming the hero in this story that’s not you, or you’re becoming this other hero and so music has a role in that storytelling. But when you’re playing multi-player it’s a little bit like watching football on television. I want to hear music when I’m watching a TV show that’s dramatic, but I don’t want to hear the football game scored – I just want to watch the football game. So if you watch a football game on television and it’s four hours long, you don’t hear music during it, and you don’t miss it... and it’s still really exciting [with just] the sound effects… you get a real visceral feeling.

You’re not expecting to have the football game to be scored. I think it would make you feel weird if as soon as the other team were about to score, some band started playing that indicated ‘Hey, somebody is about to score’. That’s not the expectation and I don’t think it enhances the experience. It seems sort of out of place. But that doesn’t mean there aren’t some kinds of multiplayer, social gaming modes that might not be enhanced by music, and we’re investigating that right now. I’m actually excited about some of the ideas that might happen here. I’m not convinced yet, we’ll see.

B: What would it take to convince you?

M: You know what it’d take? As we experiment with it I’ll be experiencing it myself and I’ll be playing and then seeing how the music actually affects me as I’m playing and how the music responds to what’s happening and if I say ‘hey this is really cool, but now that I’ve played 10 of these in a row, it’s stopped being cool’, it won’t happen. But if there’s enough interest even after repetition, or there’s enough variation or randomness in what happens, I think it could be something we do.

B: So MMO’s seem to be looking to that kind of randomly generated or procedurally generated music because people spend 100’s and 100’s of hours in them. There was a presentation at Austin GDC just the other week where they presented their way of recombining sounds that never kinda gets boring – and I guess I saw parallels in the ambient and room sounds in Halo – but it also looked to me like that sort of thing could be adapted for Halo’s multiplayer.

M: Yep, I think that would be interesting and I am definitely open to exploring those areas. This is what’s fun about game development is that there is a ton of stuff we just haven’t figured out. I know that there’s stuff around the corner that’s going to be better than what we’re doing now and I look forward to either seeing somebody else do it or if we contribute to it, that’d be great.

B: In another presentation from the 08 Austin GDC, Jason Page from Sony Europe’s R&D group, said he’s like to hear more “Phillip Glass” sound tracks in games as opposed to John William.

M: *laughs* Well you know what, when Jason says something like that, and I don’t disagree with him just so you know, but as soon as 20 games come out that all sound like Phillip Glass we’ll be pining for John Williams. The thing I would say is that…I want every creator of a game to not copy what’s successful. I think everybody who creates a game should try and re-imagine whatever it is they’re doing in their own creative image. And then there will be uniqueness even if something sorta reminds you, or it feels like a genre thing…

Sometimes I’ll sit there and go, you know what, I just want to put piano here, and it’s because I just haven’t heard piano in a while… but then if I suddenly hear a bunch of other stuff and “oh now they’re using Piano too”… I love piano, that was my major instrument, but if people are just using it because other people are using it…

For example, the cool aleatoric 20^th century sounding music that my friend did for Bioshock, if everybody starts doing that they’re doing it because they think Bioshock was successful… But he did it because it was something that came from within him that he wanted to do because he thought it would be cool. But as soon as everybody starts copying it, it’s not unique.

If you’re creating something you should create it from something that’s within you and not because you are copying something else. That doesn’t mean that what you come up with isn’t influenced by whatever you’ve grown up being influenced by, but you should never parrot or copy or do something just because this niche hasn’t been mined yet…

And the thing is that, certainly a lot of people though, ‘well there’s nothing unique about Halo – Sci Fi story, FPS, hasn’t that been done a million times?’ Yeah it has been done a million times, however…we approached it and tried to do something that hadn’t been done before even though in a lot of ways it was something that had been done before. It’s the difference between something that’s iconic and something that’s stereotypical.

B: In the conclusion of my thesis, what I’m going to end up saying is that there’s potential in all these areas and what I’d like to see is not any one way of it dominating, but having a whole bunch of different way for whatever is appropriate. And I guess that’s what you were hinting at; if that’s right for the game then go ahead, use John Williams, if it’s more appropriate, do the granular thing.

M: Absolutely, totally. Although I would never say use John Williams. You probably know this, I used to be a commercial composer and I used to write jingles for a living. And even when I was writing jingles if somebody said “you need to sound like The Beatles” or “you need to sound like Mozart” that would always be a little bit depressing to me because I felt like now I have to satisfy my client by copying Mozart. But instead of copying Mozart…I had played enough and studied enough Mozart that I went, “Hmm, I wonder how Mozart would try to solve this musical problem for this commercial.” So to some extent I would channel Mozart… what was fun was I came up with pieces that I enjoyed and that, to me, feel ‘Mozarty’. And I didn’t actually go to a piece of Mozart and just flip a few notes around until it became original. If you say, John Williams has this big bombastic and wonderful orchestral style, okay fine, what would you do with the parameters of a big bombastic orchestral style. *With this, Marty illustrated a point by singing a slight variation of the Star Wars Darth Vader theme involving some slightly changed notes, to which he expressed disparagement.* Yeah great. There’s nothing original there, you just flipped the notes around.

Next time, I ask Marty about what areas of the sound and music he wasn’t happy with in Halo 2 and he talks about the inclusion of music from outside sources such as the rock band Incubus. We also talk about musical Leitmotif and whether or not Marty likes to use that particular technique.

Marty O'Donnell in Interview - Part 4

In this, probably the longest single post of the series, Marty explains the relationship between music and location as well as how much of the music he composes in response to the level. Then, in perhaps my favourite part of the interview, Marty answers my question about why he didn’t (and still doesn’t) allow players to change the volume of the music and sound in the Halo games, resulting in a passionate response about the issue of player control vs. designer control.

Ben: So, the way Halo has used music… it seems like music is triggered based on location or events. So you crest the top of a hill and that particular string piece plays or something like that. I’m interested to know, how much of the level did you know was going to be there when you started writing and when you started recording?

Marty: I’m not sure if I could say what the percentage is… but the majority of the music is written because of what’s happening in the levels. Sometimes I’ve already made what we call music tags, tags that work a certain way or have certain variations within them or certain intensities, I know I can go from here to here. I have these music pieces that are built that will respond in certain ways and I’m building them while I’m watching what the level designers are doing.

But then before I actually implement the music or finish even recording the music, the levels are pretty much done and scripted. So we have what we call trigger volumes. Events will be triggered based on what the player’s doing, or the location of the player, or how much time the players spent. You can pick a trigger and we can trigger something off of it. Pick some event that can happen in real time and we can anticipate it and put in a condition that will be aware of that thing.

So any of those triggers I can decide, this would be a perfect point to start a piece of music, stop a piece of music, change a piece of music, overlay some other piece of music, you name it.

And because I’m sitting and working with the designers all the time, basically, I sit with them and say ‘what are you intending to do in this level, what is the emotional journey you want the payer to have from here to her to here, [etc]’ and then we just keep working that out and that’s when the music is finalized, sometimes it’s composed from scratch from those discussions, sometimes it’s music that I’ve already sort of already anticipated would be useful and I try using it in certain situations. It’s probably half & half in terms of how much music is composed prior to the level being completed and how much is composed after the level is completed and designed.

B: I also read in an interview with you, that you deliberately decided not to let the player fiddle with the volume of the music and the mix. I thought that was a really interesting choice.

M: That goes all the way back to Halo 1, you’re right.

B: And I didn’t even know as well, I’d never gone to look for it until I had started my analysis for it, and tried to turn it up so I could hear it over the noise of a tank. Is that choice related to you want to have that composer control?

M: *laughs* Probably, I think my genetic makeup is ‘I’m a composer’ but I like technology, I love games and I play games and I help design games, but… yeah, my DNA is as a composer and composers are primarily control guys. Even more than that is that I want people to have a really great experience. It’s as though I was a chef and, look, I’m making a really great meal and if the first thing somebody does when I serve them is they pour ketchup all over it or salt all over it, I’m like ‘wait a minute! You’re not tasting what I prepared. Now, if someone tastes it and goes ‘wow this tastes like crap, I need ketchup’ then fine. What I didn’t like was people, especially game players, have gotten into a habit that they should have control over all these things and basically I’m saying, you know what, no – you shouldn’t. What you should do is, play my game and if you hate it you should return it. *laughs* You should stop playing it. But I don’t want you to return it because you poured ketchup on it right away.

“If the very first thing people do, and which I’ve seen people do, is they change the mix before they even play through the game… my suggestion was, hey I have no problem giving people control over the entire mix of the game, that’s fine, I want to give you control of the entire mix of the game, however I’d like you to earn that right *laughs* and I would like the technology to be… play the game my way once, and then you unlock the mixing console and now you can go back and play the game again… but at least I know at least one time you heard it and experienced it the way I like it… I’m not even saying it’s the right way, I’m just saying this is what my vision was. So, what I don’t want-- people go ‘hey you know I had a really crappy audio experience’ and then I say ‘really?’ And they go ‘Yeah, well of course I turned the sound effects all the way up and I couldn’t hear the music’. It’s sorta like, well that’s not the experience I gave you so… On none of the games I’ve been able to convince the technologists at Bungie that we have to have a mixing console for the fans that is only implemented after they have completed the game. I would love to have that, in all honesty I think that would be ideal because then at least I know they would be experiencing the game the way I like it, and I hope they like it, and then they can play around with it to their hearts content.

But it’s sorta like going to a movie theatre and saying “I always like dialogue louder, so even before I watch this movie I want the projectionist to turn the dialogue up”, and that’s just not the way we see movies. And here’s the problem, as a passive audience member in a movie you’re just are expecting that you’re going to hear the mix that people wanted you to hear, and most of the time you say, wow either I loved that movie and I love the mix or I didn’t, but you don’t have this expectation that you’re gonna be able to control that. I don’t think it’s necessarily think it’s a good thing for game-players to think that they should have control over every parameter of a game.

You might as well say ‘hey, if you want to, change everything that’s red to green – because you prefer green. …And no one says the artists are control freaks, even though, trust me, artists are way more control freaks than composers, but that’s just my opinion. *laughs* But no ones expecting that you should be able to choose your own colour palette, unless you’re doing something like Little Big Planet which is a game which is all about the sand box, which is all about you creating your own characters and lighting and fog and textures and you name it. But basically you’re still giving players a controlled palette that designers have already decided… here’s the sandbox, and here’s the parameters we believe are fun to play in, go play.

Anyway, that’s a long answer to your question but that’s why I did that thing about the mix. Very, very few people have complained to me about not being able to mix Halo games their way. I have had some people complain and I apologise I say well that just wasn’t my intent and if you didn’t enjoy it then you didn’t enjoy my mix, and I understand that but…

B: Wow, that’s hard to believe that anyone would complain.

M: Well no I’ve had it don’t worry. *laughs* And I apologised, but at least I know they are complaining about… they don’t like my vision of the audio. They’re not just having a bad experience because they happened to set a fader badly. To most players they don’t know the difference between ‘I intended them to have this experience’ or ‘they inadvertently caused the experience to the bad’ all they remember is the bad experience.

B: Yeah that’s interesting. I had a funny experience with Bioshock at the end of last year, I put it in and I started playing and I went “The music is too loud!” because I couldn’t hear anything else, so I was really glad I could turn it down. So I guess that’s the trade-off, you’ve got to be confidents that you’ve set it all right.

M: Yeah… I mean it’s like… I make the trailers and so forth too for Halo games and no one thinks twice about… we work forever on a the final mix, right. If you’re seeing a linear trailer you’re seeing the mix… no one has the expectation that you’re going to watch a trailer and turn the music up or the dialogue down or turn the effects up or whatever and change the mix… I don’t think it’s necessarily the right expectation to have in a game. Certainly if they have that expectation it’s because they’ve been forced to use it too many times which means that’s a failure of the audio designers because they haven’t done good mixes.

Now, the caveat to that is, I believe the tradition started especially on the PC because, back in the day when you played games on PC’s everybody had different sound cards, and the audio designers could not know what the persons final experience would be, they had no clue. So they had to give all those faders and switches and sliders to people because they knew everybody’s experience was going to automatically be different. So, for me, I understand why it started, I think it’s just a bad habit. And to some extent I think it’s gotten game designers and audio designer people for games a little sloppy because they just feel like, ‘we can just throw a pile of music in there and not have to think about mixing it in, and they can throw a bunch of sound effects in and have all the sound effects normalized and unnaturally loud because they figure ‘well, people will just turn it up and down according to their preference.’ And I think that’s just not how you mix anything.

I would rather actually spend the time trying to do as good a mix as I possibly can, and then having the fans say ‘I love the game, I loved the audio’ or ‘I loved the game but the music was too soft, or the music was too loud.’ And I learn from that, most people didn’t like the mix or whatever. So far I’ve been OK.

But you see what I’m saying? What happens with gamers is they tend to think, well this is an interactive medium and that means that as the player I should have control over everything… and to me I’m thinking, you know what, no, not really… the game designer is making a game and trying to make a cool experience for you and there are going to be a lot of interactive parts to the game but it’s still… I don’t know if you’re religious at all… but it’s like Calvinism, the idea that there is a sovereign God, or there’s man’s free will, right. Well, in the game universe, the game designer is basically like God, but we want the player to believe they have free will, believe they are truly making choices but really we’re sort of God in the background saying ‘yeah you’re gonna have this illusion of free will but we know where you are, we know what you’re doing and you’re never going to get through that door no matter what you do *laughs* but here’s 5 other doors that you can choose any order you want to.”

B: Yeah, I guess the skill is in… well its sorta negative connotations to it, but manipulating the player.

M: No, no, we manipulate, no doubt about it. And frankly think about it, well you can think about it however you want, let me put it this way. You have a tribe of people. No technology at all. And everyone sits around the campfire and everyone can tell a story. Well, after a couple weeks, that tribe of people is gonna say, hey lets have Fred tell the story tonight because he always tells good stories. Just because we all have the ability or we all have the right to tell our own story… some of us are just not that good at telling stories. So certain people in the tribe become the storytellers and become the shamans, and that’s just the way people are.

Just because we say ‘hey, here’s a game and you can tell your own story’, well guess what – people don’t really want that because… probably most people tell kinda boring stories… they still want to be entertained, and to be entertained you have to have someone who’s entertaining behind it.”

Yeah that’s my philosophy, trust me I’m not necessarily right, it’s just what I think. *laughs*

In the next part, I ask Marty about musical scoring in multiplayer and why he doesn’t think it’s currently necessary, and what it would take to convince him otherwise.

Marty O'Donnell in Interview - Part 3

Between the last post and this one, Marty suggested that I go to GDC (which I would love to do one day) before we return to talking about music. This time, Marty mentions how close he’s coming with his own musical approach to realising a high level of ‘granularity’ in the music. I ask whether you can get away with more if you’re aiming for an ‘electronic’ sound rather than orchestral, and Marty talks about Rez and the potential for more ‘synaesthesia’ in games.

M: We were talking about granular synthesis or granular approaches to generating music… I would say this: I have come closer with each one because technology has been getting better, technology being that I can have more simultaneous voices and more synchronization under my control.

I’ve had ambient tracks that have had lots of different random looping elements in them that I can overlap which causes entirely different pieces to play… when I write it and dissemble it and put it in the game engine, I call it glue because I know I can glue more traditionally linear pieces together on either end with some of this more ‘gluey’ kinds of pieces that tend to be less rhythmic and stuff like that…

So in Halo 3 I was calling it “micro glue”, just for myself. And these were just individual pitches, just individual sustained notes on strings, or just an individual string note, in all sorts of different pitches so I could just generate individual notes to overlay or connect or do what I needed to and it could come in based on what people were doing…

So it would change what the piece was doing. I could essentially compose on the fly by knowing “well I know this piece is in Dm, so I could use a high E string note or a high D or an F or something and it wouldn’t clash, and I could bring that in here if this event happens… and that could sustain and then a new piece could start that’s maybe in F. So I could modulate from Dm to F because I’m also connecting it using just a single sustain or something. There’s a lot of fun just playing around with that stuff, knowing I could overlap all sorts of stuff and have it actually qued to actions that either the player does or things that we know are scripted in the game – ‘If this happens or the player gets here, this event will happened, and simultaneous to that even being triggered we trigger this new piece of music. And the new piece of music might be something as simple as just a sustained string note. But you’ll never hear that by itself, cause there will already be something going along with it…

B: It sounds like you’re actually getting quite a high level of control over instruments and notes, if you can sequenece just a single instrument and sustained notes and things, it sounds like it’s getting quite close to that granular approach.

M: It is. I’m starting to get around to around to more granular approaches. I don’t want to go completely granular… it’s one thing to record a live violin section playing a nice high F sustained note and making sure that the loop itself, that the sample has a good attack and a good release and really good loops in between so I can have it sustain as long as I want – that’s a good sample on a sampler, right? But that’s just one note. I can’t play a really good string melody using just samples. We’re getting better at it, but it still doesn’t compare to actually having the string section play that melody. The difference between what I hear when I have a string section play the melody and what I have when I’m playing samples on the keyboard playing a string melody is still really, really [large]. When I use midi performances I have to use all sorts of parameter controls just to get it even close to what a live group sounds like when they play, or just even a live soloist play. At this point I don’t have that kind of processing power to have that great of a midi playback engine inside a game engine. It could happen eventually, I’m not saying it couldn’t and I’d love to be there when it starts happening, but I still think we’re a ways off on that one. It’s not the number one priority for game engines. *laughs*

B: I was going to ask in relation to the performance aspect, a lot of the covenant and alien sounds use a lot of synths and electronic sounds. Can you get away with a more random, more granular approach with synths if you’re after that sort of alien approach?

M: Yeah, actually I think there would be nothing… you could have an entire game sorta like, have you played Rez?

B: No I’ve seen it, but never played it. I really want to play it.

M: Yeah it’s a wonderful game, it’s essentially a shooter game with all vector and Tron like graphics – like the old Tron movie. What’s really cool is that all the sound effects and everything you do is all synchronized… basically it’s all trance music that’s being played. It’s trance music that only has certain elements that play back and every more that you make is in sync with the basic beat of whatever is happening, and all the effects that you hear are musical pitches. And because they’re always in synch, while you’re playing you’re creating a piece of music on the fly that is completely in sync with what you are doing. It’s very, very effective. And because it’s this otherworldly Tron-like universe that you’re in, it works great.

B: That whole synaesthesia game genre is really interesting, and I think a lot of the ideas from that inspired my own thinking in my thesis, like what would it take to get that relationship between the music and the visuals in an FPS.

M: Yeah actually I think that’s a great place to explore… what I find is that… it works great for electronica and that sort of genre. It’s like its just tailor made for that kind of thing, but as soon as you’re doing something, like ‘this needs to have a more orchestrally scored, epic feel to it’, you struggle with that. And as soon as you say, hey you know what I love trance or I love electronica, this is perfect, but if you want to move away from that then you find it’s a lot harder to implement things that way and move into a different sort of genre of music. At least that’s my experience.

Next, Marty explains the relationship between music and location as well as how much of the music he composes in response to the level. Then, in perhaps my favourite part of the interview, Marty answers my question about why he didn’t (and still doesn’t) allow players to change the volume of the music and sound in the Halo games.

Spore 'n' Eno

You’ve gotta have an angle. I don’t remember exactly where I learned that, but I must have picked it up somewhere in between the beginning of 2^nd year uni and the end of 3^rd. If you’re aiming for a high distinction mark – to stand out from the crowd, you always need an angle; a thesis to your writing, if you will. I’d be willing to go so far as saying that sometimes (not all the time) it’s better to have a slightly suspect angle than to try and go without having anything useful to say at all. And that sounds like common sense, but it’s a lot harder to follow than you might assume. Of course, sometimes the winning option is saying nothing at all, but we don’t always have the luxury of that. In this instance I actually did have that luxury and yet forced myself to get this out anyway. Call me crazy and all that jazz. So after playing Spore for long enough to reach the space stage again this week, I can say in all honestly that I have no angle on Spore. I don’t even know if it’s possible to have one.

I was going to focus on Spore’s music because it’s in line with my own experience and interests – this blog is supposed to be somewhat inclined towards talking about music in videogames, rare as that may be. And my thesis topic was the potential of generative and interactive music in videogames after all. So believe me when I say that having no angle is not for a lack of trying. I wanted to fault it for imperfect execution, but it’s not exactly a spectacular failure. I also want to take it to task for being under-ambitious, but given all I’ve learned about the problems and issues surrounding using generative music in videogames (from people like Marty O’Donnell) I’m having difficulty even saying that. Spore and its soundtrack reached for something unique and ended up with… what exactly? Perhaps a videogame version of the most average pop album you can think of. Lady GaGa or anyone you’ll find on the front page of the iTunes store, perhaps. Think any band’s ‘sell out’ album where it’s clear they’ve gone and gotten a really great producer and a pile of cash… but there’s just no heart to it anymore (whatever the hell that’s supposed to mean). Spore went and sold its soul to The PadSynth Devil.

Take for example the ‘creature’ stage. If your little sprogling is the social type, whenever you do a song and dance to win over the hearts and minds of some fellow creature, a semi-circle bar (reminiscent of the Molyneux Fable 2 emote system) appears and you perform some rote social actions. Along with this bar and it’s filling up motion comes a glissando upwards noise that follows its trajectory. The sound it makes would not be out of place in a track on a CD in a hippy/new age shop’s $2 bin.

I want to scream, “Eno, stop pulling your punches!” I’m no devoted Eno listener, but where oh where has the avant-garde composer gone? Someone help - he’s been subsumed between layers of ambient looping spacey sounds! Where are the piercingly edgy tones of a broken speaker feeding back, lovingly sculpted into a unique sonic event? Where’s the see-saw sample that sounds like it’s audibly protesting its very existence? Why this mess of mellow in the middle of the spectrum? Audiences are ready for more.

Or are they? Honestly though, it’s not even valid to say that the music is bad. It works really well… but it puts me to sleep. I didn’t even realize it while playing all day, but when I put on some bangin’ Justice in the afternoon to do some quick cleaning and cooking I was suddenly the most wide awake all day. Does Spore really never aim to transcend its middle-of-the-afternoon public radio aesthetic? Or is it just the easiest (only?) way of overcoming the inherent problems with out of rhythm actions starting musical motifs? I refuse to believe that a consequence of generative music has to be that the listener is put to sleep. There is always going to be a dangerous temptation to incline towards more ‘mellow’ sounds to offset the ‘dissonance’ when music happens out of time, but surely the entirety of the game should not be spent in one musical register.

It’s certainly largely unswerving in the space stage and the reverb comes thick and fast. In all honesty it comes more ‘slow and low’ but there’s still a hell of a lot of it. Place any building in a colony and you’ll hear your race’s ‘leitmotif’ (which I might add you can semi self compose, but not share as a ‘creation’ as there’s simply not enough potential variation), drenched in a good two to three seconds of reverb. And that’s when the ‘instrument’ itself doesn’t already have a long and sustained tone.

Another way of thinking about it is in terms of answering the question “what can be made to work in a videogame?” Was there ever any question that reverb soaked synth, lots of low resonant sounds, plenty of sustain and long attack times would largely work in an interactive, generative environment?

The really disheartening fact is that if you thought all of this “sounds like it would be a brilliant realization of many of the ideas espoused in Ben’s thesis” then you would be right… but it’s also really not. And I’m seriously struggling to articulate why. Maybe it’s the influence of the rest of Spore which while highly ambitious also seemed to peter-out when it came to the crunch. Or maybe I’m just a pedantic contrarian and what I’m asking for would be horrible in reality. That’s a very real possibility – and if even Eno can’t do it what chance do the rest of us stand? But I suspect it’s not going to be impossible for long. And that’s about all I think I have to say about it.

Marty O'Donnell in Interview - Part 2

Marty O’Donnell is a man on the bleeding edge, being responsible for the music and audio vision for the stratospheric Halo trilogy from Bungie studios. I wanted to try and find out what this leader in the field of music and sound for games had to say about the subject and get some input for my then to be completed thesis. Around August of this year I embarked on a campaign of multiple emails to multiple addresses to somehow get in contact with Marty for an interview. After what seemed like a longer period than it probably was, finally in October a scant week before I had to hand in my thesis I spoke to the man himself via phone. Even though I approached him essentially out of the blue, Marty held no qualms about chewing my ear off for close to 90 minutes, to my own delight of course, and it is clear that music for videogames is a subject about which he is fiercely passionate.

In this, the second of the series, I ask Marty about his views on using a ‘granular’ live assembled music approach for videogame music, and he explains both why he doesn’t like that approach (for the time being) and what his own approach looks like.

Ben: I guess what’s most deceptive is, because Halo’s levels are designed so that there are multiple ways through it, but you go from point A to point B, you end up hearing all the music in the same order and maybe that contributes to the sense that you hear the same thing every time through. But I never got bored of the music in Halo either, so it’s obviously doing its job.

Marty: Halo 2 was a while back for me now, but I think the first level is the space station, right? If you got to the last encounter in the space station just before the end cinematic, pay attention to when the music actually starts, what the music does, how the music changes based on how you perform and then how the music actually seamlessly goes into the cinematic music, which is a more linear piece because I’m scoring something that’s cinematic… If you were to play through that sequence several times, try to go really fast, try to go slow, try to see what happens when the music in the level and the music in the cinematic go together, you’ll actually get several different kinds of recordings out of that.

Now it’s not like an entirely different experience, that’s not the point… it is still like, ‘that is the piece of music that plays there’, but it’s not a linear piece because different sections don’t happen in the same order and the way the music leads into the cinematic… it doesn’t [just] cross-fade it actually plays simultaneously with the cinematic music but it plays in sync with it in a way that changes depending on how you get into it.

B: So what would be your reaction to someone that wanted to do away with that sort of authorial control you have by mixing it up, and says ‘well lets set some parameters, and if the player health gets to this, do this’ (obviously that’s a simple way of doing it) do you think you do lose something if you take that step back?

M: Well you know it’s interesting because, as someone who’s worked with computer generated music or computer controlled music… and as you know in the music business and creating music, we use digital for everything… I remember back when there were music programs that were… basically lets generate music algorithmically. And I remember hearing a lot of that stuff and playing with all those things and… because I think I’m probably more of a traditionalist when it comes to what music I like, and what music I think actually evokes emotions and what music actually speaks to me, I think the composer still makes better choices than just a combination of random events.

Even when I was using algorithmic music programs, out of an hour’s worth of stuff that was generated I would find maybe 30 seconds of it that was actually interesting. So that’s my problem with that. I think that you might get some fun stuff, but I don’t think you get stuff that really speaks. Most of that stuff doesn’t end up telling a story musically, and I still think that the power of music is a storytelling power, and I might be wrong about this, but I still think that a composer tells stories better than computers do.”

However, if something is repetitive, even if it’s really, really great and you keep hearing it over and over again, and it plays back the same way every time – that is to me where boredom set in or it just starts getting annoying. Which is why when I hear games that use linear music that always plays back identically and starts to get repetitive, and loops... when I notice where the loop is I get really annoyed. That’s something I’m trying to eliminate. I really don’t want people to hear where the loops are.

B: Michael Chion wrote in the 90’s about audiovisual relationships in film and he talked about how if you strip the music off a piece of image, and you just place a selection of random other songs… some pieces of music work better with others. He said that in a few pieces… you will get a few moments of almost serendipitous synchronization between the audio and the video, so I guess where I see the potential for the live generated, granular, building the music up from individual notes, is the potential to pick out those points of synchronization and specifically hit them with the moment that you really want.

M: I would say… I have not been one of the guys who advocates the tiniest granular approach. There are some music engines and music for games approaches that’s very, very granular and right down to the individual sample level. ‘This will interact in this way’ and ‘this will interact this way’, it’s all possibly midi controlled, note generated everything.

There’s a couple reasons I don’t like that, number 1, I lose some of the fidelity that I like, I lose some of the live performance that I think is still essential… A midi flute performance just does not compare to a live flute player performing a melody. I don’t like walking away from something that has a giant history of success, so I’ve never really been an advocate of the high granular approach to writing music for games.

That doesn’t mean it’s the wrong way, it’s just that it’s my preference. The biggest problem I have with scoring a game is that if there is a sequence or an encounter or a moment or whatever it is… the most important moment is how it begins and then how it ends. Because it’s a game and there’s a human being interacting with it, what I don’t have any perfect knowledge of is exactly how long the entire experience is going to last.

So what I decided early on was, I can control when it begins, I can control when something ends, musically, what I need to do is keep the middle section malleable. And if I can do that without people knowing that it’s being malleable – so if they have a 2 minute experience some place then they get a 2 minute piece of music and they’re happy, but if somebody else play it for 5 minutes they get a 5 minute piece of music and they’re still happy because the beginning, middle and the end all correspond with what they wanted the experience to be, then I feel successful.

So it’s all about variations on the beginnings, on the ends, and being malleable with the middles. I keep trying to advance the way that stuff is manipulated musically and it’s sort of an interesting puzzle for me. I really enjoy it, I like doing it, I like composing music that I can kind of dissemble and say “what are different ways of telling the same musical story but making the middle something that is sorta indeterminate.” I don’t know if that makes sense but that’s the system.

In the next post, Marty talks about how close he’s coming with his own musical approach to realising a high level of ‘granularity’ in the music. I ask whether you can get away with more if you’re aiming for an ‘electronic’ sound, as opposed to orchestral and Marty talks about Rez & the potential for more ‘synaesthesia’ in games.