Audiosurf - Breakfast of Champions

I’ve been playing Audiosurf recently, and it struck me that buried deep within my thesis was a nice little bit of theorising about the game. So I've chosen to reprint it here, slightly edited, for the convenience of anyone who can’t be arsed to wade through my multiple thousands-of-words thesis and pick out the good bits (probably most people).

____________________________________________________________________

Audiosurf was the work of primarily by one person, Dylan Fitterer, and was released on the Steam digital distribution platform in February 2008. Audiosurf requires music to play – it takes your music collection, and creates a 3D track based upon features of the music which is then navigated by the player who, depending on the game-mode, collects coloured blocks that visually correspond to the music. The game ostensibly provides a way to ‘ride your music’ as the game’s tag-line suggests[1] - a feat of musical gameplay that is operating on a rather different level to a game like Guitar Hero. It’s also a great step towards overcoming some of the widely acknowledged problems with games like Guitar Hero - many critics have noted that the strength of a music game is largely subject to how good its track listings are[2]. Alec Meer says,

…we were all playing Guitar Hero and wishing we could stick our favourite music into it. Audiosurf says “fuck it, why not?” and provides the scaffolding of a game around it[3]

Audiosurf’s particular implementation of representing and performing music in a game does however come with a number of its own disadvantages. Firstly, the way the three dimensional track is generated by the program is fixed and determined by a set algorithm[4]. In an interview with Ars Technica, the developer Dylan Fitterer commented on the way that the algorithm turns the song into a three dimensional track, saying;

…when the music is at its most intense, that's when you're on a really steep downward slope, like you're flying down a rollercoaster in a tunnel. When the music is calmer, that's when you're chugging your way up the hill, watching that peak in the distance you're going to reach.[5]

The experience of playing the game itself is where I personally find the major innovations of Audiosurf as well as its major problems. When surfing a song the game’s analysis algorithm has pre-determined the majority of the course’s parameters from the musical elements contained within the recording. Some aspects of the course are determined from relatively transparent musical parameters – the track’s length corresponds directly to the length of the song and the contours of the course are derived from reasonably straightforward aspects such as volume and dynamics. In music with a strong steady beat, the track will often appear to undulate along beneath the player’s ship character in time with the rhythm of the song. The comprehensible translation of the music into visuals, or lack thereof, is where I encounter the main problem of Audiosurf.

In the examples outlined above, the relationship between music and the visuals (the track environment) is clear and direct, making sense to the player and allowing for a pleasurable and organic merging of knowledge of the song with knowledge of the corresponding Audiosurf track. This is a significant aspect of the appeal of the game as much community discussion goes on about the suitability of tracks for surfing[6]. Indeed the process works effectively on the macro structural scale, however a core component of Audiosurf is a ‘match 3’ type block collection game, where the block placement – called ‘traffic’ by the game – is generated from the rather more musically ambiguous parameter of “volume spikes”. The developer, Dylan Fitterer, describes the process saying

…whenever there's a spike in the music, the intensity of that spike determines the block's color. So the most distinct spikes, like a snare drum, that would tend to be a red block, a really hot block. If something is a little more subtle, like a quiet high hat, that would be a purple block, which is worth less points.[7]

This kind of relationship between music and visuals or environment becomes, musically at least, increasingly murky on this micro level as a sheer ‘spike’ in volume is no guarantee that a listener would make the corresponding connection to what they are hearing. Indeed the issue of what a listener actually perceives about a song when listening to it is much, much more complicated. Albert S. Bregman, author of the comprehensive text ‘Auditory Scene Analysis: The perceptual organisation of sound’ coined the term “stream” for what he identified as an audible cognitive process which was lacking adequate terminology. Bregman’s research noted a significant distinction between the cognitive process of the grouping of sounds that ‘go together’[8] from what might be distinguished as pure ‘sounds’. He notes that, ‘A series of footsteps, for instance, can form a single experienced event, despite the fact that each footstep is a separate sound.’ He also makes a musical comparison, saying that,

A soprano singing with a piano accompaniment is also heard as a coherent happening, despite being composed of distinct sounds (notes). Furthermore, the singer and piano together form a perceptual entity – the “performance” – that is distinct from other sounds that are occurring.[9]

Kieron Gillen writing for Rock, Paper, Shotgun says that

The problem with Audiosurf is that the concentration you take to really make the block game work is entirely the opposite of what you need to do to feel the music. The two parts of the game can tug at each other a little...On one hand, a zone game. On the other, a high-speed sorting puzzle.[10]

What I believe that Gillen has identified here is the inherent disjunction between what the musical listener focuses on when listening to the song, and what the game makes the player focus on. I suggest that this phenomenon is somewhat analogous to Ian Bogost’s term ‘simulation fever’. The concentration Gillen identifies as being necessary for successful play means that the player is acutely aware of block placement, largely determined by the volume spikes mentioned earlier.

I would argue that simply focussing on volume spikes is not adequately representative of the music to withstand the scrutiny that a player applies to it. I propose that, in a situation of high concentration on music, a more complex system is needed, one which addresses the issue of how a listener perceives a song. Admittedly, this is a daunting prospect and one inevitably encounters certain apparently insurmountable barriers to rendering onscreen what any one particular person is most likely to concentrate on within a song at any one time, needing as it would to take into account personal differences and background as well as individual musical training. However, the fact remains that this process is undertaken by humans themselves leads me to believe that a more accurate model is possible. When listening we can (and do) lock onto a number of particular elements of a song – the melody, a catchy lead rhythm or hook – and this is not always represented visually on screen. While Audiosurf often wonderfully represents the underlying kick-drum rhythm, especially if it is prominent, it will rarely pick up and single out an element like the aforementioned melody or hook unless it stands out in a particular way – namely through sheer volume.

Guitar Hero, in contrast, sidesteps some of these problems through both its position as a guitar game (with the player’s concentration largely limited to being focussed on the guitar) and by having a human pre-define the on screen actions the player has to undertake to ‘perform’ the song. However it does not yet allow for any meaningful input of a players own music library, and for that I am continually thankful for Audiosurf’s existence – imperfect though it may be.

____________________________________________________________________

---

[1] Wikipedia contributors, "Audiosurf," Wikipedia, The Free Encyclopedia, http://en.wikipedia.org/w/index.php?title=Audiosurf&oldid=241996378, accessed October 7, 2008.

[2] See for example, Mitch Krpata, ‘Rock Band 2: Why now?’, Insult Swordfighting, http://insultswordfighting.blogspot.com/2008/07/rock-band-2-why-now.html, accessed October 7^th, 2008.

[3] Alec Meer in ‘The RPS Verdict: Audiosurf’, Rock, Paper, Shotgun, http://www.rockpapershotgun.com/2008/03/03/the-rps-verdict-audiosurf/, accessed March 3, 2008.

[4] Thomas Wilburn, ‘Catching Waveforms: Audiosurf Creator Dylan Fitterer speaks’, Ars Technica, http://arstechnica.com/journals/thumbs.ars/2008/03/11/catching-waveforms-audiosurf-creator-dylan-speaks, accessed

[5] Dylan Fitterer in Thomas Wilburn, ‘Catching Waveforms: Audiosurf Creator Dylan Fitterer speaks’.

[6] See the comments section of any Rock, Paper, Shotgun Post tagged ‘Audiosurf’ – every single one involves readers suggesting songs that others should try: http://www.rockpapershotgun.com/tag/audiosurf/

[7] Dylan Fitterer in Wilburn, ‘Catching Waveforms: Audiosurf Creator Dylan Fitterer speaks’.

[8] Albert S Bregman, Auditory Scene Analysis : The Perceptual Organization of Sound, 2nd MIT Press paperback ed., Cambridge, Mass.: MIT Press, 1999, p.9

[9] Ibid, p.10

[10] Kieron Gillen in ‘‘The RPS Verdict: Audiosurf’, Rock, Paper, Shotgun, http://www.rockpapershotgun.com/2008/03/03/the-rps-verdict-audiosurf/, accessed March 3, 2008.

Fun and Loathing in Las Vegas Washington D.C.

Why is this not as funny as it should be?

Lately, I’ve been trying to play Fallout 3 in a way that doesn’t get me as angry and annoyed as the situation I described in my previous post about the game. I’ve started using a few mods that both improve the interface and make the ‘karma’ system more opaque. While welcome, sadly the changes are a case of ‘too little too late’ to enable anything more than a “meh” feeling about Fallout 3 in general. While the rage has died down, I can’t resist but add my .02c on what’s still bothering me.

Mostly it’s to do with the moral judgments the game makes and the attendant Karma system. One reviewer (I fail to recall who, perhaps Alec Meer?) in their write-up of Fallout 3 mentioned that whatever character you chose to play, the Fallout 3 world would resist the temptation to punish you harshly or judge you for your actions. Having only first hand experience with Fallout 2, I can’t speak for the whole series, but this seems quite consistent with the feel established by the first two games. If you wanted to kill random people and take their stuff, you’d be feared, naturally, but that was about it. There was really no judgment about what kind of person you were. Similarly, the villains were primarily cartoon-stereotype individuals, deluded madmen (and women) or brutish bullies. They conformed to typical archetypes that allowed them to transcend cliché and enter the realm of intelligent satire.

In Fallout 3 there is a character called Mr. Burke. He is a suit who works for a very rich man and apparently revels in nihilistic destruction. Let’s imagine how this character would have appeared in Fallout 1 or 2. He would probably be suited, probably comes with a bodyguard or two and would fit into the “madmen killer” stereotype. You understand him, and that he is meant to be a stereotype. You know that he is unstable, a ruthless killer, but also otherwise completely rational. He sees life as something that is made more precious by death, and so seeks to spread death in order to re-value life in this post-apocalyptic wasteland. You understand his twisted and flawed logic, his broken personality and know that there is some kind of rationale for why he is this way, something in his past, perhaps. Maybe he just read too much Nietzsche, either way, you are never supposed to believe that he is inherently bad or evil, just misguided in the extreme. There is no presiding moral outrage here unless you choose it yourself, just impartial acknowledgement of the fact that he must be stopped or he will continue to kill. End of story.

Let’s contrast this with the character of Mr. Burke in Fallout 3. His voice and character says he’s some of these things, and his dialogue is written to make sense this way… but his voice actor (voiced I am sure by the same person who did the The Speaker for The Dark Brotherhood in Oblivion) inflects in his voice that, “I am slimy and evil”. His voice drips with “evil” and “nastiness”. You should hate me and be outraged by my actions, it says. This is not Fallout. This is too ham-fisted for Fallout. If Mr Bourke is to be serious, it should be more like the disturbing style of Heath Ledger as ‘The Joker’ than cartoon-villain Jack Nicholson.

The difference lies in that while my earlier characterization of Mr. Burke was intentionally simplistic, he was a parody of and commentary on that simple-minded, 1950’s aesthetic. His character manages to pose deep questions while being simplistic when the latter is a po-faced affair that lacks the same semi-aware playfulness that says “I know I am a stereotype and I’m playing up to it”. In short, Mister Burke pisses me off for taking himself too seriously. It would be fine if he did, except there is no accompanying pay-off in additional depth. He should be scary to be around, intimidating and edgy.

This picture also has nothing to do with this post.

I think most of this is a result of a combination of voice acting, writing and animation, and Fallout 3 to me seems to end up making judgments without asking the real questions with regards to the Mr. Bourke character. Instead of traditional morality being destroyed by the nuclear bombs that wiped out so much else of civilization, it has been dragged into the apocalyptic wasteland without a care for how well it fits. It fails to capture the levity of the original titles and loses the accompanying ability to contrast the whimsical with the horrific. It would be all too easy to blame this on the transition from cartoonish 2D sprites to full 3D models, but I think that’s a bit of an easy out.

I don’t quite know what the answer to this problem is. I have the feeling, though, that there’s something they could learn from Left 4 Dead’s ability to blend being constantly surrounded by death in the apocalypse (the serious) with the decidedly upbeat characters. The aesthetic at least seems closer to my taste than Fallout’s. That said, however, the beautiful wasteland remains pretty much the only place in Fallout that I can enjoy without cringing at something or someone every other second.

Thesis is complete!

And thus ends another chapter of my life; that of being an undergraduate student. I printed, bound and submitted my thesis last Friday and I have been Walking on a Dream since.

I plan to turn the more compelling parts of it into a blog series, either here or elsewhere, and elaborate a bit on my findings. I'll also make it publicly available at the same time, which should be around mid-November when I get it back from being marked. In the intervening time, you can read the abstract of my Thesis:

Abstract

Unlike traditional artistic endeavours such as literature, painting or sculpture, videogames and their creation, according to Janet Murray, are still in an incunabular period. Various efforts have been made to view videogames in light of other media such as film and narrative while few have yet to address, specifically, ways in which videogames present unique opportunities for expression. This thesis draws upon a number of authors to identify areas unique to videogames, and examines the implications for the employment of music within them. After examining the case for videogame uniqueness, the thesis looks to the current musical paradigm within videogames and, finding it somewhat lacking, offers a critique of the paradigm. A number of games that do, however, break from tradition and utilise music in exceptional ways are then discussed and their potential for adoption in future games is assessed. The final component of the thesis is an investigation into the use of music within the Xbox videogame Halo 2 (2004) through discussion with the composer, Martin O’Donnell, and an analysis of the music and sound of the game. In the process I discover that the game uses music in a way similar to the dominant paradigm, while also exhibiting a musicality within the in-game sound effects and level ambience. The result is a ‘soundscape’ style approach well suited to attaining both the emotive power of linear compositions as well as a closer relationship between music and visuals, seemingly a ‘best of both worlds’ videogame musical approach.

In a previous life I was...

...a mashup artist.

So here's a link to my critically acclaimed* mashup album, Demon Says...A Mashup Album.

A little bit of background on the album, all done in an attractive, bullet point style:

• > I made it for a 20 credit point unit at Uni (It was half of a semesters load worth of study)
  
• > It was made in around 4 months
• > It is a thematic song-by-song remixing/reworking/mashing-up of the entire Gorillaz album 'Demon Days' (hence Demon Says... Geddit? like Simon Says... okay you're right it wasn't all that clever but it stuck.)
• > It features other such prominent artists such as: Coldplay, Fergie, Will Smith, Radiohead, Rhianna, Lady Sovereign, Justin Timberlake and plenty of other unwitting victims.
• > It got me a distinction. Nuff said surely.
• > Oh yeah, it's free cause if I tried to make money from it I'd be hunted down and killed.
• > Please try and listen to the whole album - there's supposed to be a thematic continuity and I think some of that idea survived the process. The ending songs do get a bit weird, but I like weird, and so do you - go on, admit it.
• > Some song titles are appropriaely nerdy (e.g. the final tracks' title 'Exits are north south, east and Dennis'. If you don't get it, I'm not explaining it to you)
• > I really enjoyed making it, and some of it remains some of my best work. I hope you like at least some of it too.
  

Go nuts people, but don't tell the authorities on me, K?

*by critically acclaimed I mean that I, speaking critically, am full of accolades for it.

The attraction to collaboration

I haven't posted in a while, mostly because I'm on holidays and I'm lazy, but also because I've been playing lots of Oblivion on PC. I bought a new graphics card last week (a Gigabyte 8800GT 512mb) which is awesome and makes PC games fun again. Hooray for generous tax returns to poor students! Anyway, what I wanted to quickly mention was something entirely unrelated. I've been thinking about it for a while and I just wanted to jot down some thoughts about it and ask the audience.

Collaboration. From Miriam-Webster online:

1 : to work jointly with others or together especially in an intellectual endeavor
2 : to cooperate with or willingly assist an enemy of one's country and especially an occupying force
3 : to cooperate with an agency or instrumentality with which one is not immediately connected

Yes, that's the sound of this post heading in a completely unexpected direction. Collaboration is everywhere. It's in Music (any band who ever played together), it's in Writing (any writer and their editor/s), its in Videogames (in any multi-person development or mod team) and it can be in blogging (like RPS, etc).

I'm bored of doing stuff by myself - for starters, its never as good as when I work with someone else as what always seems happen is we end up inspiring each other and produce something better as a result. In my second year at Uni we had a subject called Collaborative Project in which we formed inter-disciplinary groups of artists from music, drama, electronic arts and fine arts to put on a short 20 minute work at the end of the semester. The groups were as big as 15-20 people, and easily became unwieldy, with decision making being done mostly on a basis of whoever actually bothered to turn up to the class (which was about 5-10 of us regularly =P). Anyway, it was a great time and one of the best things about such a large group was that you could do something very specialised within the group. My contribution, other than often helping facilitate discussions, was a piece of musical improvisation that used live digital signal processing to make lovely ethereal sounds.

A screenshot of my Max/MSP patcher - a visual programming environment made for digital signal processing, and my first musical love. The bottom box was used to 'draw' a spectral filter that was applied to white noise, resulting in some awesome sounds.

I've uploaded a short sample of a very early version of what it sounded like, just for fun, here. (1.9mb, 4:10) The final performance was... OK - and our supervisory lecturer really enjoyed it, so we obviously did something right. Good marks too.

The point that I wanted to to get to was that I want to collaborate more. The Internet is possibly the most powerful communication enhancing tool out there and can facilitate some great works... so where are they all? Have you got a great idea you want to collaborate on? A Mod idea? A musical work you want to do? Something you want to write about that you want input from others on? If so, tell me about it! Collaborate with me, people! I want to give you my time!

So in conclusion, consider this an official invitation to sequester me for all sorts of strange collaborations. Got an idea, drop it in the comments or send me an email and we'll talk! I'd love to hear from anyone and everyone, and I'd also love to hear your success/horror stories of collaborations. If they've turned you off them, why? They can be ever so much fun.

Markov Chain Madness

I must warn you - The following text is generated by a Markov chain. Inspired by Mr Quixoric Engineer on Twitter I decided to have a little play around with some Markov generated text, using my paper on virtual reality and the FPS as the input text. Needless to say, the results were stunning. Better than anything I could have written

In the virtual from raw ingredients often bringing conventions and game means stepping into a phrase, everything about the largest city and this peculiarity of the map’ and those far away hills I made my own experience. Finally, I believe there is a large body of wanderlust through a mountain ranges – with his main argument down into a picture of wanderlust through the player, relating a taxonomy of a children’s game Oblivion is Alchemy, which they may employ an unreal image to the massive draw distance – with ancient ruins and reaction. At a reading for engaging with possibility.

I really like the bit where it says "Oblivion is Alchemy". Of COURSE! Why didn't I think of that? It makes so much sense now. So there you have it. Play with Markov chain generative text here. After my initial experiments, the markov generator came up with this last piece of inspired literature, which makes a kind of scary sense.

In this research paper aimed at discussing some of the group ‘Iraq and Afghanistan Veterans of America’ spoke to the author of the issues raised in the Baudrillardian sense, that the developers chose to employ an invisible barrier as their means of keeping the game that I wish to explore the world, I began to enacting the role playing game into a world in which to move about and interact with the same topics in greater depth. One addition to the FPS solely owns the ‘first person perspective’ videogames have not been an independently delineated genre – rather existing simply as a popular first person viewpoint gameplay conventions and tropes set in place by a desire to explore the world, I experience a heightened level of ‘simulation fever’ because, as just mentioned, I have been problems for the player. For example, not having the time to fill in the case in most, if not all games, and is clearly room for telling stories in other places however there was one small barrier between me and those far away hills of promise, and it was later released as a focus on state-of-the-art graphics with other role playing game, The Elder Scrolls IV: Oblivion. I then made a small assertion that virtual comes from the surface, stay in touch with reality” . Ryan says

…the meaning of virtual ice climbing I was rather excited. I had, after all, spent the last several minutes on the DVD to store the extra environment or not having enough space on the other the virtual as fake and the other side.

When I finally did reach a section of the game as encouraging the practice of stopping to smell the flowers.

A different aspect of the many ‘skills’ that a play can employ within

Think I could get away with Markov chaining my whole Literature Review? Feel free to link me some of your excellent finds in the comments.